For the past few years we have been lamenting the state of benchmarks for mobile platforms. The constant refrain from those who had been around long enough to remember when all PC benchmarks were terrible was to wait for the release of Windows 8 and RT. The release of those two OSes would bring many of the traditional PC benchmark vendors space into the fray. While we're expecting to see new Android, iOS, Windows RT and Windows 8 benchmarks from Futuremark and Rightware, it's our old friends at Kishonti who are first out of the gate with a cross-OS/API/platform benchmark. GLBenchmark has existed on both Android and iOS for a while now, but we're finally able to share information and performance data using DXBenchmark - GLB's analogue for Windows RT/8.

As the name difference implies, DXBenchmark uses Microsoft's DirectX API while GLBenchmark relies on OpenGL ES. The API difference alone makes true cross-platform comparison difficult, especially since we're comparing across APIs, OSes and hardware - but we at least have the option to get a rough idea of how these platforms stack up to one another. There are a lot of improvements expected with Windows Blue later this year from a platform optimization standpoint from the ARM based SoC vendors, so I wouldn't read too much into any of the Android vs. Windows RT comparisons of the same hardware (even though some key results end up being very close).

While GLBenchmark 2.7 doesn't yet take advantage of OpenGL ES 3.0 (GLB 3.0 will deliver that), it does significantly update the tests to recalibrate performance given the advances in modern hardware. Version 2.7 ditches classic, keeps Egypt HD and adds a new test, T-Rex HD, featuring a dinosaur in pursuit of a girl on a dirt bike:

Scene complexity goes up tremendously with the T-Rex HD benchmark. GLBenchmark has historically been more computationally bound than limited by memory bandwidth. The transition to T-Rex HD as the new flagship test continues the trend. While we see scaling in average geometry complexity, depth complexity and average memory bandwidth requirements, it's really in the shader instruction count that we see the biggest increase in complexity:

GL/DXBenchmark 2.7: T-Rex HD Compared to Egypt HD Benchmark
  Increase in T-Rex HD
Average Geometry Complexity +55%
Average Depth Complexity +41%
Average Texture Memory Bandwidth Requirements +41%
Average Shader Instruction Count +165%

T-Rex HD should benefit from added memory bandwidth, but increases in raw compute performance will be most visible. Given the comparatively static nature of memory bandwidth improvements, scaling shader instruction count to increase complexity makes sense.

Just as before, both GL and DXBenchmark 2.7 can run in onscreen (native resolution, v-sync enabled) and offscreen (1080p, v-sync disabled) modes.

The Android and iOS versions retain the UI of their predecessors, while DXBenchmark 2.7 introduces a Windows RT/8 flavored take on the UI:

There will be a unified database of scores across both GL and DXBenchmark once the latter gets enough submissions.

The low level tests are comparable between GLBenchmark 2.5 and 2.7, only results from the new T-Rex HD benchmark can't be compared to anything GLBenchmark 2.5 produced (for obvious reasons).

Now time for the exciting part. The usual suspects from the iOS and Android worlds are present, I didn't include anything slower than a Tegra 3 given how low T3 scores in the T-Rex HD test. From the Windows RT camp we've got Microsoft's Surface RT (Tegra 3) and Dell's XPS 10 (APQ8060A/Adreno 225). The sole 32-bit Windows 8 Pro representative is ASUS' VivoTab Smart (Atom Z2560/PowerVR SGX 545). Finally, running Windows 8 Pro (x64) we have Microsoft's Surface Pro (Core i5-3317U/HD 4000) and the Razer Edge (Core i7-3517U/GeForce GT 640M LE).

As always, we'll start with the low level results and move our way over to the scene tests:

GL/DXBenchmark 2.7 - Fill Test (Onscreen)

GL/DXBenchmark 2.7 - Fill Test (Offscreen)

Looking at the fill rate tests, we have the first indication of how Intel's HD 4000 graphics compares to the best in the tablet space. Unconstrained, Surface Pro delivers a fill rate of over 2x that of the 4th generation iPad. NVIDIA's GeForce GT 640M LE delivers nearly 3x the fill rate of the iPad 4.

The Mali-T604 in Google's Nexus 10 finds itself in between the iPad 4 and iPad mini, while Tegra 3 ends up faster than both the Clover Trail and Qualcomm Windows RT platforms. It's interesting to note the big difference in fill rate between the Nexus 7 (Android/Tegra 3) and Surface RT (Windows RT/Tegra 3). You would think that driver maturity would be better on Windows for NVIDIA, but assuming this isn't some big API difference it could very well be that Tegra 3 on Android is more mature.

GL/DXBenchmark 2.7 - Triangle Throughput (Onscreen)

GL/DXBenchmark 2.7 - Triangle Throughput (Offscreen)

The gap in geometry performance between Intel's HD 4000 and Imagination Tech's PowerVR SGX 554MP4 grows to over 2.5x. Surface RT and the Nexus 7 switch positions, and grow a lot closer than they were in the fill rate test. Qualcomm's Windows RT platform remains at the bottom of the list, and Intel's Clover Trail remains disappointing in the graphics department.

GL/DXBenchmark 2.7 - Triangle Throughput, Fragment Lit (Onscreen)

GL/DXBenchmark 2.7 - Triangle Throughput, Fragment Lit (Offscreen)

Increase the complexity of the triangle test and things don't change all too much.

GL/DXBenchmark 2.7 - Triangle Throughput, Vertex Lit (Onscreen)

GL/DXBenchmark 2.7 - Triangle Throughput, Vertex Lit (Offscreen)

Moving on to the scene tests, we have the first look at the current landscape of T-Rex HD performance on tablets. When Egypt HD first came out, the best SoCs were barely able to break 20 fps with the majority of platforms delivering less than 13 fps. In the 8 months since the release of GLBenchmark 2.5, the high end bar has moved up considerably. The best tablet SoCs can now deliver more than 40 fps in Egypt HD, with even the latest smartphone platforms hitting 30 fps. T-Rex HD hits the reset button, with the fastest ARM based SoCs topping out at 16 fps.

GL/DXBenchmark 2.7 - T-Rex HD (Onscreen)

GL/DXBenchmark 2.7 - T-Rex HD (Offscreen)

Looking at the offscreen results, we finally get what we came here for. Intel's HD 4000 manages to deliver 3x the performance of the PowerVR SGX 554MP4, obviously at a much higher power consumption level as well. The Ivy Bridge CPU used in Surface Pro carries a 17W TDP, and it's likely that the A6X used in the iPad 4 is somewhere south of 5W. The big question here is how quickly Intel can scale its power down vs. how quickly can the ARM guys scale their performance up. Claiming ARM (and its partners) can't build high performance hardware is just as inaccurate as saying that Intel can't build low power hardware. Both camps simply chose different optimization points on the power/performance curve, and both are presently working towards building what they don't have. The real debate isn't whether or not each side is capable of being faster or lower power, but which side will get there first, reliably and with a good business model.

To put these results in perspective, the GPU in the Xbox 360 still has around 3x the compute power of what's in the iPad 4. We're getting closer to having current (soon to be previous) gen console performance in our ultra mobile devices, but it'll take another year or two to get there in the really low power devices. Surface Pro is already there.

The rest of the players here aren't that interesting. Everything from the Tegra 3 to the old A5 in the iPad mini performs fairly similarly when faced with the same display resolution (1080p). Despite standings in some of the lower level benchmarks, Qualcomm's aging APQ8060A platform in the Dell XPS 10 (Windows RT) manages a healthy performance advantage over Intel's Atom Z2560 - both aren't particularly exciting parts from a graphics performance standpoint however. It's interesting to note just how close Surface RT and the Nexus 7 are here, given that they are running different OSes, using different APIs but powered by the same Tegra 3 SoC.

The only other scene test we have is Egypt HD, which is a known quantity these days. The only new bits are the inclusion of the Windows players using DXBenchmark:

GL/DXBenchmark 2.5 - Egypt HD (Onscreen)

GL/DXBenchmark 2.5 - Egypt HD (Offscreen)

Everyone's performance looks a lot better under the Egypt HD test, which is of course the motivation for creating the T-Rex HD test. Also interesting to note is the Apple/Intel gap shrinks a bit here, now the advantage is only 2x. It's important to put all of this in perspective. If your ultimate goal is to be able to run a shader heavy workload like T-Rex HD, then most of the tablet platforms have a long way to go. If Epic's Citadel demo release on Android is any indication however, there's a lot that can be done even with the mainstream level of performance available on smartphones and tablets today. Identifying and delivering the best performance at whatever that sweet spot may be is really the name of the game here, and it's one that the ARM folks have done a great job of playing.

I'm very curious to see how these graphs change over the next two years. I don't suspect Haswell will shift peak platform power down low enough to really be viewed as an alternative to something like an iPad, but with Broadwell (2014) and Skylake (2015) that may be a possibility. The fact that these charts are even as close as they are, spanning 7-inch tablets all the way up to full blown PC hardware, is an impressive statement on the impact of the mobile revolution.

POST A COMMENT

83 Comments

View All Comments

  • Milliamp - Monday, April 01, 2013 - link

    I think this is interesting for sure. I think another interesting thing to throw in would be a 5 - 7 year old Dell PC with a cheapish dedicated GPU.

    I'm also curious how something like Razer Edge compares to the Xbox 360 or PS3 but it would be harder to measure. Something like that serves no purpose than to satisfy my curiosity of how far behind mobile graphics are from other platforms but I am sure other people wonder the same thing.
    Reply
  • designerfx - Monday, April 01, 2013 - link

    how is this "behind"? The battery life of the Razer edge is defined by the power cable aka nonexistent. Performance wise it's more telling that the Razer performs better than the surface.

    I think things are pretty much exactly where they should be. I agree that if you have a unified benchmark it would be a good time to test performance across the board.
    Reply
  • extide - Monday, April 01, 2013 - link

    Razer Edge is faster than Xbox 360/PS3 in both CPU and GPU. Reply
  • amb9800 - Monday, April 01, 2013 - link

    Quick correction: Surface Pro specs should read "i5-3317U" instead of "i3-3317U." Reply
  • IntelUser2000 - Monday, April 01, 2013 - link

    It's still not a good test because the Razer Edge's 640M LE should outperform the HD4000 by 50-100% or more, but is behind the iGPU in this one. Even worse than the 3DMark benches. At least 3DMark can be a rough representation of shortly upcoming titles. Reply
  • Kevin G - Monday, April 01, 2013 - link

    It wasn't a surprise to seeing the Surface Pro and the Razer Edge leading the pack in these tests. The question was how much of a lead they'd carry over more mobile centric ARM parts. These numbers don't bode well for Intel. ARM SoC's are looking to further improve CPU and GPU performance this year while in the same thermal envelope as is Intel. The problem is that Intel will still be at a higher power consumption. By the time Broadwell and/or Skylake reach consumers, the battle for control of the mobile market may have already been won by ARM. Part of winning the battle is showing up for the fight. Reply
  • MadMan007 - Monday, April 01, 2013 - link

    You forget Baytrail Atom SoCs - many CPU architectural improvements, and Intel HD-based graphics. Those will more than 'show up for the fight'. Reply
  • Kevin G - Monday, April 01, 2013 - link

    It'll show up at the cost of performance. It is unclear how that new architecture will stack up with ARM's offerings when it arrives. That is still 8 months away with a new generation of ARM and PowerVR chips due out in the same time frame. Reply
  • DanNeely - Monday, April 01, 2013 - link

    Baytrail is rumored to have a 4core IGP; if it's the same core as in IVB it'll be ~25% as fast as what the SurfacePro is showing give or take variations in clocking/etc. That's between the Nexus10 and iPad4 in performance. While fast enough to be competitive, it's almost certain to be eclipsed by GPUs in this years new arm SoCs. Reply
  • londiste - Monday, April 01, 2013 - link

    why not bode well? 10w haswell will be a pretty good improvement over the current 17w ivy, given that the performance will be same or better (which so far seems it will be). new generation atoms are also coming in autumn (rumored to have a significant gpu overhaul, probably intel's own hd).

    on arm side, things are going well, but they are also starting to run into the same tdp problem on both cpu and gpu front - with substantial improvements in performance, tdp will rise.

    and at the same time, both performance and tdp gaps are still significant. interesting times.
    Reply

Log in

Don't have an account? Sign up now