Driving the Retina Display: A Performance Discussion

As I mentioned earlier, there are quality implications of choosing the higher-than-best resolution options in OS X. At 1680 x 1050 and 1920 x 1200 the screen is drawn with 4x the number of pixels, elements are scaled appropriately, and the result is downscaled to 2880 x 1800. The quality impact is negligible however, especially if you actually need the added real estate. As you’d expect, there is also a performance penalty.

At the default setting, either Intel’s HD 4000 or NVIDIA’s GeForce GT 650M already have to render and display far more pixels than either GPU was ever intended to. At the 1680 and 1920 settings however the GPUs are doing more work than even their high-end desktop counterparts are used to. In writing this article it finally dawned on me exactly what has been happening at Intel over the past few years.

Steve Jobs set a path to bringing high resolution displays to all of Apple’s products, likely beginning several years ago. There was a period of time when Apple kept hiring ex-ATI/AMD Graphics CTOs, first Bob Drebin and then Raja Koduri (although less public, Apple also hired chief CPU architects from AMD and ARM among other companies - but that’s another story for another time). You typically hire smart GPU guys if you’re building a GPU, the alternative is to hire them if you need to be able to work with existing GPU vendors to deliver the performance necessary to fulfill your dreams of GPU dominance.

In 2007 Intel promised to deliver a 10x improvement in integrated graphics performance by 2010:

In 2009 Apple hired Drebin and Koduri.

In 2010 Intel announced that the curve had shifted. Instead of 10x by 2010 the number was now 25x. Intel’s ramp was accelerated, and it stopped providing updates on just how aggressive it would be in the future. Paul Otellini’s keynote from IDF 2010 gave us all a hint of what’s to come (emphasis mine):

But there has been a fundamental shift since 2007. Great graphics performance is required, but it isn't sufficient anymore. If you look at what users are demanding, they are demanding an increasingly good experience, robust experience, across the spectrum of visual computing. Users care about everything they see on the screen, not just 3D graphics. And so delivering a great visual experience requires media performance of all types: in games, in video playback, in video transcoding, in media editing, in 3D graphics, and in display. And Intel is committed to delivering leadership platforms in visual computing, not just in PCs, but across the continuum.

Otellini’s keynote would set the tone for the next few years of Intel’s evolution as a company. Even after this keynote Intel made a lot of adjustments to its roadmap, heavily influenced by Apple. Mobile SoCs got more aggressive on the graphics front as did their desktop/notebook counterparts.

At each IDF I kept hearing about how Apple was the biggest motivator behind Intel’s move into the GPU space, but I never really understood the connection until now. The driving factor wasn’t just the demands of current applications, but rather a dramatic increase in display resolution across the lineup. It’s why Apple has been at the forefront of GPU adoption in its iDevices, and it’s why Apple has been pushing Intel so very hard on the integrated graphics revolution. If there’s any one OEM we can thank for having a significant impact on Intel’s roadmap, it’s Apple. And it’s just getting started.

Sandy Bridge and Ivy Bridge were both good steps for Intel, but Haswell and Broadwell are the designs that Apple truly wanted. As fond as Apple has been of using discrete GPUs in notebooks, it would rather get rid of them if at all possible. For many SKUs Apple has already done so. Haswell and Broadwell will allow Apple to bring integration to even some of the Pro-level notebooks.

To be quite honest, the hardware in the rMBP isn’t enough to deliver a consistently smooth experience across all applications. At 2880 x 1800 most interactions are smooth but things like zooming windows or scrolling on certain web pages is clearly sub-30fps. At the higher scaled resolutions, since the GPU has to render as much as 9.2MP, even UI performance can be sluggish. There’s simply nothing that can be done at this point - Apple is pushing the limits of the hardware we have available today, far beyond what any other OEM has done. Future iterations of the Retina Display MacBook Pro will have faster hardware with embedded DRAM that will help mitigate this problem. But there are other limitations: many elements of screen drawing are still done on the CPU, and as largely serial architectures their ability to scale performance with dramatically higher resolutions is limited.

Some elements of drawing in Safari for example aren’t handled by the GPU. Quickly scrolling up and down on the AnandTech home page will peg one of the four IVB cores in the rMBP at 100%:

The GPU has an easy time with its part of the process but the CPU’s workload is borderline too much for a single core to handle. Throw a more complex website at it and things get bad quickly. Facebook combines a lot of compressed images with text - every single image is decompressed on the CPU before being handed off to the GPU. Combine that with other elements that are processed on the CPU and you get a recipe for choppy scrolling.

To quantify exactly what I was seeing I measured frame rate while scrolling as quickly as possible through my Facebook news feed in Safari on the rMBP as well as my 2011 15-inch High Res MacBook Pro. While last year’s MBP delivered anywhere from 46 - 60 fps during this test, the rMBP hovered around 20 fps (18 - 24 fps was the typical range).


Scrolling in Safari on a 2011, High Res MBP - 51 fps


Scrolling in Safari on the rMBP - 21 fps

Remember at 2880 x 1800 there are simply more pixels to push and more work to be done by both the CPU and the GPU. It’s even worse in those applications that have higher quality assets: the CPU now has to decode images at 4x the resolution of what it’s used to. Future CPUs will take this added workload into account, but it’ll take time to get there.

The good news is Mountain Lion provides some relief. At WWDC Apple mentioned the next version of Safari is ridiculously fast, but it wasn’t specific about why. It turns out that Safari leverages Core Animation in Mountain Lion and more GPU accelerated as a result. Facebook is still a challenge because of the mixture of CPU decoded images and a standard web page, but the experience is a bit better. Repeating the same test as above I measured anywhere from 20 - 30 fps while scrolling through Facebook on ML’s Safari.

Whereas I would consider the rMBP experience under Lion to be borderline unacceptable, everything is significantly better under Mountain Lion. Don’t expect buttery smoothness across the board, you’re still asking a lot of the CPU and GPU, but it’s a lot better.

Achieving Retina Boot Camp Behavior & Software Funniness
Comments Locked

471 Comments

View All Comments

  • Fx1 - Monday, June 25, 2012 - link

    Whine Whine Whine.

    In the UK we have the best warranty going thanks to UK Regulations and EU protection. Plus 3 year free on education discounts of which i have bought 2 MBP's and never set foot in a university in my life.

    Just because in NZ you don't have that protection then blame your gov for not passing legislation.

    Also why bitch about Ram? just put the 16GB in and then your good for the next 10 years i expect. £13 a year cost
  • Spunjji - Tuesday, June 26, 2012 - link

    You're so full of shit it's bursting out at the seams. The NZ government have imposed much better consumer warranty protection than we have here in the UK. You might know that if you weren't a fucking cretin who spouts off nonsense before thinking. The fact that you seem to think it's his fault that he lives in NZ where Apple give shitty support just shows what a jingoistic little fanboy you are.

    That's also an interesting "£13 a year" theory. It'd be neat if it wasn't a complete and utter crock. If more than 10% of these laptops are used regularly for 10 years, Apple pixie magic included, then I will eat my goddamn chair. Maybe you should have gone to university, they could have slammed some sense into your tiny little skull.
  • Solidstate89 - Sunday, June 24, 2012 - link

    "Apple’s impact on the industry has already been felt. The threat of Apple bringing Retina Displays to its entire lineup forced ASUS’ hand and gave us 1080p IPS panels in the new Zenbook Primes."

    Isn't that a little fucking arrogant to say with absolutely no proof to back that claim up? You know better than anyone how much design work goes into producing these products, and just how LONG the process takes. You really think in just a few months time of rumors, ASUS could have completely revamped their product line-up? Or maybe they were just doing it, because they had already started with the higher resolution displays on their tablet line-up? Or maybe they were even trying to match the absolutely superb and high density display that Sony has been using in their Vaio Z series for all these years.

    Claiming just the RUMOR of Apple doing it was UNDOUBTEDLY the influence behind ASUS' IPS, high resolution screens is baseless bullshit. And you know it.
  • Iketh - Sunday, June 24, 2012 - link

    what i don't get is how people could care so much to get pissed off like you... get a hobby!

    and i agree 100% with Anand's statement... you're blind to not agree, and i'm a PC developer
  • wfolta - Sunday, June 24, 2012 - link

    ASUS may have decided to carry higher resolution displays from their tablets to their laptops, but Apple beat them to the punch with the retina iPad and the rumors of a retina Macbook have been around since at least that far back.

    Considering most manufacturers have reduced pixel density over time, and considering it was Apple that pushed the retina display first to the iPhone, then the iPad, I think the burden of proof is on you to say that these netbook makers were just itching for higher resolution.

    In fact, most of them stopped at the stupid 16:9 1080p (1920x1080) resolution because that's what HD TVs have and they've wanted to tout their ability to play BluRay's at full resolution. There was no concept of making a beautiful display: it was all about a marketing checkbox. And that's what they guy is saying.
  • kmmatney - Sunday, June 24, 2012 - link

    I'm a windows developer - have been for almost 20 years now. I also travel a lot so all my development is done on a laptop, across several virtual machines. I'm currently using a 2007 Dell 17" 1920 x 1200, and have not upgraded yet, waiting for another windows 1920 x 1200 laptop to surface. I've run out of hope now - if Apple comes out with a 17" version of this I'll have to try and convince my company to buy one. I've never used a Mac in my life, but will pay the "Apple tax" to get what I need hardware-wise. I currently have about 20 VMs packed onto a Samsung 256GB 830, so I'm good with storage space.

    The Windows laptop makers just don't get it - not everyone watch's movies all day - some of us want to use our machines for real work and want to have extra vertical space. Is it that hard to make a 16:10 screen? An ugraded screen is a "3D" screen - really?! The "True-life" screen on my old 2007 Dell is still better than anything you can get nowadays - I've directly compared it with co-corkers brand-new laptops.
  • KPOM - Sunday, June 24, 2012 - link

    Is it possible that Apple's solution for making the rMBP work will scale to the new iPhone? In other words, does it offer Apple a way to increase the screen size without causing a break in compatibility between older and newer apps?
  • gstrickler - Sunday, June 24, 2012 - link

    The iPhone and iPad already handle the HiDPI displays, Apple deployed Retina displays there first. The scaling to non integer resolutions is definitely something Apple could do on the iPhone or iPad, but since it takes significantly more CPU/GPU power and affects both performance and battery life, I would not expect it anytime soon. If Apple goes to a larger screen on either device, it's likely that it will support the same total resolution as the current devices and have slightly lower dpi. If we see a 3.7"-4.3" iPhone display, I expect it to be 960x640, just like the current iPhone with Retina display.

    They already have 4 native resolutions to support on iDevices, 480x320 and 960x640 on iPhone, 1024x768 and 2048x1536 on iPad. I don't believe they'll want to increase that as it has an impact on the hardware, battery life, and every developer, so while it's possible, I think it's unlikely.
  • raclimja - Sunday, June 24, 2012 - link

    Ingadget
  • hrrmph - Sunday, June 24, 2012 - link

    This one is *extremely* close to allowing me to replace a full tower workstation and my current laptop. Apple has built a very, very impressive machine that should be a wake-up call to the industry: Move forward, or be left behind.

    Even so, it falls slightly short of my ideal specs. If Apple would make the next one a 17" or 18" version (please make it Retina, I love the foresight there :), add a 10-Key section to the keyboard, upgrade to Giga-WiFi-ac-1.3Gbs, upgrade the quantity of internal drive ports (yes plural - I can use up to 5 immediately, if they will fit), and pretty please give us a standard drive connector.

    While we're at it, can we have all of the drives running at speeds of at least 6.0Gbs? You know, like putting the drives on the current standard mSATA 6.0Gbs connectors, or even hooking them up to a better, newer, smaller, open standard connector at 12.0Gbs.

    That's a lot to ask of course, but if they 'just' managed to make it a 17" Retina display, with three drives, with standard connectors, and WiFi-ac-800Mbs, and managed to do it this year, then I would feel they have done as much as anyone could do in in the time available.

    If they miss out on delivering a 17" model this year, then it becomes more of a wide open competition where almost anyone could deliver a competitive product in the 17"+ space. After all, the display actually only has to be better than 1920 x 1200 to exceed the specs of my current laptop display.

    Right now Apple appears to have the hardware lead and a fairly big one at that. If such a 17"+ machine appears, as described in any of the scenarios above, then it would solidify Apple's hardware lead. With a move like that, Apple would firmly and aggressively cement its position in every laptop screen size category that matters. They could have a compelling position in perhaps 99% of the laptop market.

    It would also satisfy my need to have a portable large display machine that can go on infrequent, but very long trips, where I am sometimes remote, and I often need to be fairly self-sufficient and fault tolerant (ie: fix any of the inevitable data glitches myself).

    It would improve my situation very significantly if there were sufficient physical drives to have separate system, data, and auto-backup drives internally. Also, having extremely capable external ports is important. So both keeping the existing wired ones just released on the current 15" Retina model, and adding the nascent WiFi-ac wireless internal hardware that is debuting this year in non-Apple equipment, would be super helpful for enhancing my ability for doing redundant off-machine data backups - just in case the internal auto-backup gets thrashed.

    In fact if Apple were able to produce such a machine, I would be virtually forced - in a very pragmatic sense - to shell out the cash, move to the dark side, and purchase my first Apple product, ever.

    Yes, after 25 years of computing I would lose my patience with the laggards that comprise the rest of the laptop market. With deep, naughty feelings of dark misgiving, I would both splurge and switch allegiance at the same time. I wonder how many others would do the same?

    If this should come to pass, then I would inevitably wonder if Apple's real intentions are as wicked as I fear: To eventually try to lock all of us as 'sheeple', into a proprietary hell on Earth. You know... like Sony's thankfully failed attempts to force 'works only here' on us. Umm... let's see... MMC Memory Sticks, anyone?.

    Well, back to Apple: Just sayin'... I would do it... even though I know better.

    Hey... isn't Microsoft supposed to be getting back into the hardware business in a big way?

    What do you think the chances are of a company like Microsoft and a machine like that...

    So you're telling me there's a chance... *YEAH!*

    -

Log in

Don't have an account? Sign up now