GPU Performance Using Unreal Engine 3

In our iPad 2 review I called the PowerVR SGX 543MP2 Apple's gift to game developers. Apple boasted a roughly 9x improvement in raw GPU compute power over the A4 into the A5. The increase came through more execution resources and a higher GPU clock. The A5 in the iPhone 4S gets the same GPU, simply clocked lower than the iPad 2 version. Apple claims the iPhone 4S can deliver up to 7x the GPU performance of the iPhone 4, down from 9x in the iPad 2 vs. iPad 1 comparison. Why the delta?

The iPad 2 has both a larger battery and a higher resolution display. There are 28% more pixels to deal with on the iPad 2 vs the iPhone 4S and 9x vs 7x actually works out to be a 28% increase. The lower clocked GPU goes along with the lower clocked CPU in the 4S' version of the A5 to keep power consumption in check and because the platform doesn't need the performance as much as the iPad 2 with its higher resolution display.

Mobile SoC GPU Comparison
  Adreno 225 PowerVR SGX 540 PowerVR SGX 543 PowerVR SGX 543MP2 Mali-400 MP4 GeForce ULP Kal-El GeForce
SIMD Name - USSE USSE2 USSE2 Core Core Core
# of SIMDs 8 4 4 8 4 + 1 8 12
MADs per SIMD 4 2 4 4 4 / 2 1 ?
Total MADs 32 8 16 32 18 8 ?
GFLOPS @ 200MHz 12.8 GFLOPS 3.2 GFLOPS 6.4 GFLOPS 12.8 GFLOPS 7.2 GFLOPS 3.2 GFLOPS ?
GFLOPS @ 300MHz 19.2  GFLOPS 4.8 GFLOPS 9.6 GFLOPS 19.2 GFLOPS 10.8 GFLOPS 4.8 GFLOPS ?

GLBenchmark continues to be our go-to guy for GPU performance under iOS. While there are other reputable 3D benchmarks, GLBench remains the only good cross-platform (iOS and Android) solution we have today.

The performance gains live up to Apple's expectations (Update: our original 4S for Egypt/Pro were incorrect. We had two sets of graphs, one internal and one external - the latter had incorrect data. We have since updated the charts to reflect the 4S' actual performance. Sorry for the mixup!):

GLBenchmark 2.1 - Egypt - Offscreen

GLBenchmark 2.1 - Pro - Offscreen

GLBenchmark gets around vsync by rendering offscreen, so the 4S is allowed to run as fast as it can. Here we see a 6.46x higher frame rate compared to the iPhone 4.

It's obvious that GLBenchmark is designed first and foremost to be bound by shader performance rather than memory bandwidth, otherwise all of these performance increases would be capped at 2x since that's the improvement in memory bandwidth from the 4 to the 4S. Note that we're clearly not overly bound by memory bandwidth in these tests if we scale pixel count by 50%, which is hardly realistic. Most games won't be shader bound, instead they should be more limited by memory bandwidth.

At the iPhone 4S introduction Epic was on stage showing off Infinity Blade 2, which will have new visual enhancements only present on the 4S thanks to its faster GPU. Thus far Epic has been using GPU performance improvements to make its games look better and not necessarily run faster (although they do) since the target is playability on all platforms. What I wanted however was a true apples-to-apples comparison using Epic's engine as it is arguably the best looking platform to develop iOS games on today.

Epic offers a free license to Unreal Engine 3 to anyone who wants to use it for non-commercial use. If you want to sell your UE3 based iOS game, you don't have to pay a large sum to license Epic's engine up front. Instead you toss Epic $99 and pay royalties (25%) on any revenue beyond the first $50K. It's a great deal for aspiring game developers since you get access to one of the best 3D engines around and don't need any additional startup capital to use it. If your game is a hit Epic gets a cut but you're still making money so all is good in the world.

The process starts with UDK, the Unreal Development Kit. Epic actually offers a great deal of documentation on developing using UDK, making the whole process extremely easy. The freely available UDK can target Windows, Mac OS X and iOS platforms. If you want Android support you'll have to pay to license the dev kit unfortunately. Given how successful Infinity Blade has been under iOS, I suspect this is a move partially designed to keep Apple happy. It's also possible the Android UE3 dev kit is simply not as far along as the iOS version.

Along with every UDK download, Epic now provides the full source code to its well known iOS Citadel demo. With access to Citadel's source code and Epic's excellent (and freely available) development tools I put together a real-world GPU test for iOS.


What's that? A frame counter in iOS? Huzzah!

The test shows us frame rate over the course of a flythrough of Epic's Citadel demo. This is simply the standard Citadel guided tour but with UE3's frame recording statistics enabled. Once again, UDK gave me the tools needed to accurately profile what was going on. For developers this would be helpful in tuning the performance of your app, but for me it gave me the one thing I've been hoping for: average frame rate in a UE3 game for iOS.

The raw data looks like this, a graph of frame render times:


iPhone 4S frame time

You're looking at frame render time in ms, so lower numbers mean better performance. Notice how the iPhone 4S graph seems to remain mostly flat for the majority of the benchmark run? That's because it's limited by vsync. At 60Hz the frame render time is capped to 16.7ms, which is approximately where the 4S' curve flattens out to. The 4S could likely run through this demo even quicker (or maintain the same speed with a heavier graphical workload) if we had a way to disable vsync in iOS.


iPhone 4 frame time

On the iPhone 4 however, frame times are significantly higher - more than 2x on average. You also see significant spikes in frame time, indicating periods where the frame rate drops significantly. Not only does the 4S offer better average performance here but its performance is far more consistent, hugging vsync rather than wildly bouncing around.

The chart below summarizes the two graphs above by looking at the average frames rendered per second throughout the benchmark:

AnandTech UE3 Performance Test

The iPhone 4S averages 2.3x the frame rate of the iPhone 4 throughout our test. I believe this gives us a more realistic value than the 6x we saw in GLBenchmark. A major cause for the difference is the vsync limitations present in all iOS apps that render to the screen. On top of that, while we're obviously not completely limited by memory bandwidth, it's clear that memory bandwidth does play a larger role here than it does in GLBenchmark.

The Citadel demo by default increases rendering quality on the iPhone 4, but a quick look at the game's configuration files didn't show any new features enabled for the 4S. Chances are the version of Citadel included with the UDK was built prior to the 4S being available. In other words, the 4 and 4S should be rendering the same workload in our benchmark. To confirm I also grabbed a couple of screenshots to ensure the two devices were running at the same settings:


iPhone 4


iPhone 4S

This is actually the most stressful scene in the level, it causes even the 4S to drop below 30 fps. With the camera stationary in roughly the same position I saw a 74% increase in performance on the 4S vs the iPhone 4.

Most game developers still target the iPhone 3GS, but the 4S allows them to significantly ramp up image quality without any performance penalty. Because of the lower hardware target for most iOS games and forced vsync I wouldn't expect to see 2x increases in frame rate for the 4S over the 4 in most games out today or in the near future. You can expect a smoother frame rate and better looking games if developers follow Epic's lead and simply enable more eye-candy on the 4S.

The Memory Interface The A6: What's Next?
POST A COMMENT

201 Comments

View All Comments

  • Formul - Monday, October 31, 2011 - link

    There is about a dozen fairly similar android phones every month. As iPhone gets refreshed just once every year (this time year and a half) and has bigger market than any single android phone out there it makes sense to make the review thorough. I guess Nexus Prime and even Razer will get similar lengthy reviewsa ... as did many other android phones before that (galaxy s2 for example had about the same as this one).

    You are the one biased here. Counting the number of android articles on any gadget blog will seriously outpace the iPhone by far. You should shut up and read what you want, no one is forcing you to read this blog or any particular article on it. I dont read android articles, i dont care about android all that much. Why do you have to troll here if it does not interest you after all?
    Reply
  • Anand Lal Shimpi - Monday, October 31, 2011 - link

    I believe we did just that for the Galaxy S 2 review:

    http://www.anandtech.com/show/4686/samsung-galaxy-...

    We don't have the RAZR, Galaxy Nexus or S2 Skyrocket, but we typically do a deep dive whenever there's a new platform transition or something truly unique to look at.

    Take care,
    Anand
    Reply
  • LordSojar - Tuesday, November 01, 2011 - link

    Hmm, touche Anand, I had forgotten about how detailed the S2 review was.

    The RAZR doesn't warrant a particularly detailed review I suppose, but I think the S2 Skyrocket might warrant a detailed antenna review etc considering the transition to LTE along with, what should be a major change to the antenna and should (I think) have QC's MDM9615 or MDM8215 chips in it... which is a big deal considering those could yield not only significant speed gains along with an antenna update, but some major throughput.

    Cheers
    Reply
  • doobydoo - Tuesday, November 01, 2011 - link

    'Samsung releases a new phone that has overall better features, faster CPU, faster NAND, a different and arguably better (or at least equal) screen'

    I hope you are comparing to the OLD Samsung, in this sentence, rather than comparing to the iPhone 4S.

    The CPU/GPU performance of the iPhone 4S is not comparable to the Galaxy S2, and I would argue the features are very much superior too.

    Which, incidentally, along with the dual-antenna design, Siri, brand new camera, makes the iPhone 4S much more of a step up than the Samsung Galaxy to Samsung Galaxy S2. (The Galaxy S was about the same performance as the iPhone 4. The Galaxy S2 is slower than the iPhone 4S, therefore iPhone 4S is a bigger upgrade).

    And of course, as Anand said, they thoroughly review most phones, and as the other poster says, the Android phones come out way more often so over the same length of time they get at least as much if not more coverage.
    Reply
  • Tetracycloide - Tuesday, November 01, 2011 - link

    The Galaxy S2 released in May. The 4S released in October. The next Galaxy phone, the generation the 4S is actually going to compete with, is the Galaxy Nexus. What I'm saying is your point would hold more water (or any at all) if Samsung didn't make 2 upgrades in the space of time it took Apple to do 1 so the differential between generations is hardly an apples to apples comparison (pun intended). Reply
  • thunng8 - Tuesday, November 01, 2011 - link

    But the Galaxy Nexus isn't any faster than the S2. The main benefit is the new operating system and higher resolution screen.

    The Nexus is actually a significant step backwards from the S2 in terms of camera (likely from the samples on the prototype) and GPU (definitely since it is using the SGX540) performance
    Reply
  • doobydoo - Wednesday, November 02, 2011 - link

    Release dates will not stop phone users comparing the iPhone 4S to the Androids released in March 2012, just like they compared all the Androids over the last 17 months to the iPhone 4.

    The iPhone 4S, right now, is competing with the Galaxy S2 directly.

    The Nexus Prime, as the other comment alluded to, is a significant downgrade from the Samsung Galaxy S2. Indeed, it uses the same GPU as used in the much older Samsung Galaxy S (albeit clocked higher).

    The Galaxy S to S2 took approximately a year inbetween generations (June 2010 to May 2011), which is very much in line with the typical Apple release cycle of annually (this year was the exception). Even this year, with the extended timeline between iPhone 4 and 4S, Samsung didn't make 2 upgrades during that time. Your point would hold more water (or any at all) if it had.
    Reply
  • Tetracycloide - Wednesday, November 02, 2011 - link

    It's obvious you're not making a real comparison between current products but instead of admitting that you pretend it's a non-issue and then strangely fought that issue anyway by suggesting I'm not actually right to call you out on these shenanigans. So thanks for being obtuse about this, it makes it much easier to dismiss your position as that of an irrational fan-boy (at least it does when combined with the condescension in some of the other threads you've commented on here). Reply
  • doobydoo - Friday, December 02, 2011 - link

    'It's obvious you're not making a real comparison between current products'

    Erm, huh?

    That's EXACTLY what I'm doing.

    What YOU'RE doing is asking is to compare a phone which isn't even out yet with a phone which is. You're trying to 'pretend it's a non-issue' that the phone you say the iPhone 4S should be compared with wasn't even out when this discussion started. THAT is the definition of fanboy-ism. It's like the constant like of Android fans 'x WILL be better when it eventually comes out' - and THAT is obtuse.

    So, the only comments which can be dismissed are YOUR fanboy comments about 'x phone released in x months will be better'. Turns out you were wrong anyway, but that's irrelevant - the bottom line is you don't want iPhone to be compared to CURRENT Android phones, but instead compared to FUTURE ones.

    Rational, much?

    As it happens, it's out now, and isn't even much of an improvement on the Samsung Galaxy S2
    Reply
  • Lucian Armasu - Wednesday, November 02, 2011 - link

    The cameras seem to be about the same, according to Anand. I've actually found the SGS II camera to be a bit better in photo comparisons. Siri? I still think it's more of a gimmick for now. Maybe in 5-10 years it will be actually useful. Once the novelty expires, people will barely even use it in a few months. The same happened with FaceTime which everyone went crazy about at launch, and then they stopped using it completely.

    The CPU of the SGS2 IS faster than iPhone 4S CPU. There's no way Apple managed to get a 50% performance improvement for the same Cortex A9 chip that Samsung is using.

    I think Anand can confirm this. He wasn't very clear in the review. He was only talking about the improvements it gets over the old Cortex A8.
    Reply

Log in

Don't have an account? Sign up now