GPU Performance Using Unreal Engine 3

In our iPad 2 review I called the PowerVR SGX 543MP2 Apple's gift to game developers. Apple boasted a roughly 9x improvement in raw GPU compute power over the A4 into the A5. The increase came through more execution resources and a higher GPU clock. The A5 in the iPhone 4S gets the same GPU, simply clocked lower than the iPad 2 version. Apple claims the iPhone 4S can deliver up to 7x the GPU performance of the iPhone 4, down from 9x in the iPad 2 vs. iPad 1 comparison. Why the delta?

The iPad 2 has both a larger battery and a higher resolution display. There are 28% more pixels to deal with on the iPad 2 vs the iPhone 4S and 9x vs 7x actually works out to be a 28% increase. The lower clocked GPU goes along with the lower clocked CPU in the 4S' version of the A5 to keep power consumption in check and because the platform doesn't need the performance as much as the iPad 2 with its higher resolution display.

Mobile SoC GPU Comparison
  Adreno 225 PowerVR SGX 540 PowerVR SGX 543 PowerVR SGX 543MP2 Mali-400 MP4 GeForce ULP Kal-El GeForce
SIMD Name - USSE USSE2 USSE2 Core Core Core
# of SIMDs 8 4 4 8 4 + 1 8 12
MADs per SIMD 4 2 4 4 4 / 2 1 ?
Total MADs 32 8 16 32 18 8 ?
GFLOPS @ 200MHz 12.8 GFLOPS 3.2 GFLOPS 6.4 GFLOPS 12.8 GFLOPS 7.2 GFLOPS 3.2 GFLOPS ?
GFLOPS @ 300MHz 19.2  GFLOPS 4.8 GFLOPS 9.6 GFLOPS 19.2 GFLOPS 10.8 GFLOPS 4.8 GFLOPS ?

GLBenchmark continues to be our go-to guy for GPU performance under iOS. While there are other reputable 3D benchmarks, GLBench remains the only good cross-platform (iOS and Android) solution we have today.

The performance gains live up to Apple's expectations (Update: our original 4S for Egypt/Pro were incorrect. We had two sets of graphs, one internal and one external - the latter had incorrect data. We have since updated the charts to reflect the 4S' actual performance. Sorry for the mixup!):

GLBenchmark 2.1 - Egypt - Offscreen

GLBenchmark 2.1 - Pro - Offscreen

GLBenchmark gets around vsync by rendering offscreen, so the 4S is allowed to run as fast as it can. Here we see a 6.46x higher frame rate compared to the iPhone 4.

It's obvious that GLBenchmark is designed first and foremost to be bound by shader performance rather than memory bandwidth, otherwise all of these performance increases would be capped at 2x since that's the improvement in memory bandwidth from the 4 to the 4S. Note that we're clearly not overly bound by memory bandwidth in these tests if we scale pixel count by 50%, which is hardly realistic. Most games won't be shader bound, instead they should be more limited by memory bandwidth.

At the iPhone 4S introduction Epic was on stage showing off Infinity Blade 2, which will have new visual enhancements only present on the 4S thanks to its faster GPU. Thus far Epic has been using GPU performance improvements to make its games look better and not necessarily run faster (although they do) since the target is playability on all platforms. What I wanted however was a true apples-to-apples comparison using Epic's engine as it is arguably the best looking platform to develop iOS games on today.

Epic offers a free license to Unreal Engine 3 to anyone who wants to use it for non-commercial use. If you want to sell your UE3 based iOS game, you don't have to pay a large sum to license Epic's engine up front. Instead you toss Epic $99 and pay royalties (25%) on any revenue beyond the first $50K. It's a great deal for aspiring game developers since you get access to one of the best 3D engines around and don't need any additional startup capital to use it. If your game is a hit Epic gets a cut but you're still making money so all is good in the world.

The process starts with UDK, the Unreal Development Kit. Epic actually offers a great deal of documentation on developing using UDK, making the whole process extremely easy. The freely available UDK can target Windows, Mac OS X and iOS platforms. If you want Android support you'll have to pay to license the dev kit unfortunately. Given how successful Infinity Blade has been under iOS, I suspect this is a move partially designed to keep Apple happy. It's also possible the Android UE3 dev kit is simply not as far along as the iOS version.

Along with every UDK download, Epic now provides the full source code to its well known iOS Citadel demo. With access to Citadel's source code and Epic's excellent (and freely available) development tools I put together a real-world GPU test for iOS.


What's that? A frame counter in iOS? Huzzah!

The test shows us frame rate over the course of a flythrough of Epic's Citadel demo. This is simply the standard Citadel guided tour but with UE3's frame recording statistics enabled. Once again, UDK gave me the tools needed to accurately profile what was going on. For developers this would be helpful in tuning the performance of your app, but for me it gave me the one thing I've been hoping for: average frame rate in a UE3 game for iOS.

The raw data looks like this, a graph of frame render times:


iPhone 4S frame time

You're looking at frame render time in ms, so lower numbers mean better performance. Notice how the iPhone 4S graph seems to remain mostly flat for the majority of the benchmark run? That's because it's limited by vsync. At 60Hz the frame render time is capped to 16.7ms, which is approximately where the 4S' curve flattens out to. The 4S could likely run through this demo even quicker (or maintain the same speed with a heavier graphical workload) if we had a way to disable vsync in iOS.


iPhone 4 frame time

On the iPhone 4 however, frame times are significantly higher - more than 2x on average. You also see significant spikes in frame time, indicating periods where the frame rate drops significantly. Not only does the 4S offer better average performance here but its performance is far more consistent, hugging vsync rather than wildly bouncing around.

The chart below summarizes the two graphs above by looking at the average frames rendered per second throughout the benchmark:

AnandTech UE3 Performance Test

The iPhone 4S averages 2.3x the frame rate of the iPhone 4 throughout our test. I believe this gives us a more realistic value than the 6x we saw in GLBenchmark. A major cause for the difference is the vsync limitations present in all iOS apps that render to the screen. On top of that, while we're obviously not completely limited by memory bandwidth, it's clear that memory bandwidth does play a larger role here than it does in GLBenchmark.

The Citadel demo by default increases rendering quality on the iPhone 4, but a quick look at the game's configuration files didn't show any new features enabled for the 4S. Chances are the version of Citadel included with the UDK was built prior to the 4S being available. In other words, the 4 and 4S should be rendering the same workload in our benchmark. To confirm I also grabbed a couple of screenshots to ensure the two devices were running at the same settings:


iPhone 4


iPhone 4S

This is actually the most stressful scene in the level, it causes even the 4S to drop below 30 fps. With the camera stationary in roughly the same position I saw a 74% increase in performance on the 4S vs the iPhone 4.

Most game developers still target the iPhone 3GS, but the 4S allows them to significantly ramp up image quality without any performance penalty. Because of the lower hardware target for most iOS games and forced vsync I wouldn't expect to see 2x increases in frame rate for the 4S over the 4 in most games out today or in the near future. You can expect a smoother frame rate and better looking games if developers follow Epic's lead and simply enable more eye-candy on the 4S.

The Memory Interface The A6: What's Next?
POST A COMMENT

201 Comments

View All Comments

  • weiran - Wednesday, November 02, 2011 - link

    > If you like a different phone design, you get a Droid RAZR.

    Which is ugly as sin, and isn't really an "upgrade" from an iPhone 4.

    IMO from a design POV the most interesting iPhone competitor right now is the unreleased Nokia Lumia 800, for hardware and software.
    Reply
  • doobydoo - Friday, December 02, 2011 - link

    Hardware wise, the difference between the iPhone 4S CPU/GPU combination and Android competitors, is huge.

    If you want the best performance, I don't see any other way to turn.

    As for the software, I like most users would happily use Android or iOS - they've largely converged anyway.
    Reply
  • name99 - Friday, November 04, 2011 - link

    "If you had a 4 already your just a "moop" if you upgraded to one of these!"

    OK, so most US customers on on a 2yr plan. Upgrading for them is impractical, if for no other reason then for carrier reasons. And then you are COMPLAINING that Apple produced an upgrade that (in your eyes) gives them no reason to upgrade???

    What exactly is your problem? You're like someone who goes to a restaurant and complains "the food sucks --- and the portions are too small".
    Reply
  • dennykins - Tuesday, November 01, 2011 - link

    Hey! I have that "Introduction to Modern Optics" book too! Pretty old, but still relevant Reply
  • name99 - Tuesday, November 01, 2011 - link

    "It is admittedly curious that Apple hasn’t decided to make some other larger change to distinguish the 4S from the other two"

    There was, of course, also precious little visual change between iPhone, iPhone3G and iPhone 3GS. Likewise for plenty of other upgrades across the Apple line in the last ten years. I don't know why Anand thinks it's strange. For YEARS Apple has made the point that you buy an iMac --- and you get the what Apple thinks an iMac should be today. You don't buy an iMac XV371. You don't buy an iMac 7. You don't buy an iMac 20011. You buy an iMac --- which may or may not look like its predecessor, which may or may not have last been updated three, six or
    nine months ago.

    It's obvious that Apple wants to bring that same mindset to phones (and iPads). Screwing with people's expectations of an update in July was simply the first step in breaking the mindset of a particular schedule for upgrades. Of course there are issues that make it more difficult to do this cleanly, for example carrier involvement and the ridiculous subsidized pricing model --- which means that Apple has reason, at least for now, to keep older models around. But there are obvious advantages to Apple in switching to this model, including
    - not being forced to release SW too soon. I think we'd all agree iOS5 was released under pressure, and that iOS 5.1 will be the release 5.0 should have been.
    - Apple's been able to ignore pressure from a stream of constant Android updates for the past two years, but at some point annual upgrades may just be too slow. At that point it would be nice to have the option of minor upgrades (cf the recent PowerBook Pro upgrades), say boosting the CPU from 800 MHz to 1GHz, without it being a big deal --- something to shut up the crowd that looks at specs, while being ignored by the mass audience that does not care about specs and doesn't want Apple talking about them.

    Not that this will stop haters from claiming that "people buy iPhones for fashion"....
    Reply
  • KPOM - Tuesday, November 01, 2011 - link

    Good observation about the upgrades. Yes, it's entirely possible we'll see minor spec bumps as the iPhone has become a staple of Apple's product line. We don't notice all the the upgrades in the Android world (how many people know or even care about the differences between the Galaxy S, Galaxy S II, and Galaxy S II Skyrocket ?), partly because they happen all the time.

    Also, Apple has never been a company to change the design simply for the sake of change. They tweak a design until they "get it right," but then stick with it a while. This year's MacBook Air, for instance, is a big update from last year's (Core i5/i7, backlit keyboard, 4GB RAM standard on most models), but they didn't change the external design. It's still selling extremely well and received good reviews. I think it's the same with the iPhone. The 4S has some significant improvements, from the faster processor, better camera, and better antenna.
    Reply
  • name99 - Tuesday, November 01, 2011 - link

    I'm glad you guys adopted my suggestion to report how snappy a phone feels, based not only on CPU benchmarks but also on the speed of flash, and things like the launch time of apps.

    In the spirit of constant complaint that is the web, can I now ask that you continue to do this for all future phones, not just iOS devices.
    Yes, you have fanboi readers who care only about how wonderful their platform is; but you do also have a number of honest readers who are genuinely interested in things like how the speed of flash (including the speed of SD flash) varies across devices, and how launch times (for comparable apps) vary --- which, of course, depends on both HW and also SW/OS decisions.
    Reply
  • Drasca - Tuesday, November 01, 2011 - link

    There's a small typo that I'm surprised no one else has caught. Perhaps I am the only one that cares enough about this particular subject enough to. On the last page, 4th to last paragraph, we see:

    "we’ll hopefully see the technology mature into something more like what years of science fiction moves have promised us."

    I believe you mean movies here.

    I caught it in particular because I've been discussing amongst my friends how modern technology is the future promise of older science fiction. Oh gosh gee wiz, we have instant mobile communication across the globe, including video! We talk into our portable devices and they think for us. These devices are an extension of ourselves. In a way, we've become like the borg in that we're connected to community at large. Facebook is a form of hive mind group consciousness. Heck, we can track each other near-instantaneously and some folk are vividly aware of each other's locations.

    These are interesting times, and what has only previously been explored as supposition has become reality.

    I still want my flying cars and teleporters. Also, Holodecks and the AI behind it. Plus, the bridge of the starship enterprise D with its big comfy captain's chair and massive screen.
    Reply
  • anishannayya - Wednesday, November 02, 2011 - link

    If you want an iPhone, you really don't have a choice. Either you get the 4S or go find an Android phone. Reply
  • shashank7040 - Wednesday, November 02, 2011 - link

    Asus Eee Pad being the first tablet With Slide out QWERTY........http://goo.gl/B4rJU Reply

Log in

Don't have an account? Sign up now