Testing Notes

Before diving into the nuts and bolts of our actual review, I wanted to stop for a moment and talk about the means and perspectives on how to best test Apple’s rather unparalleled tablet. In terms of mechanical benchmarks, the path is rather straightforward – almost frustratingly so. There are only so many decent standardized benchmarks that run on the iPad Pro, and even fewer of those that overlap with other operating systems, particularly windows. This is why, as atypical and throughput-focused as SPEC is, it remains one of the better tools for determining how the hardware compares to other devices.

However when it comes to the user experience, that’s another matter. The iPad Pro is an iPad, that is also Pro. Specifically, that it has been designed and is being specifically pitched not only as a tool for the iPad’s traditional content-consumption tasks, but also professional productivity use cases as well. Document editing, content creation, photo editing, and other tasks that while not outside of the realm of a regular iPad, aren’t really its forte either.

Since there aren’t really any other serious Arm-based tablets of the iPad Pro’s caliber on the market – Android seems to slip farther and farther behind every year – for the review of this year’s significantly redesigned model, I’m opting to approach matters from the other direction: how does it compare to traditional productivity machines. This is the market occupied by the likes of the Microsoft Surface Pro and other Windows-powered convertibles; devices that have brought the Windows experience to a tablet-like form factor. And while the overlap is by no means perfect, I do feel that these sorts of devices are the standard-bearers for productivity and professionally-oriented tablets on the whole.

So at least in my eyes, the real competition for the new iPad Pros is going to be these other pro devices, rather than a smattering of large-format Arm-based tablets. And these are the sorts of devices I’m primarily going to be looking to compare the iPad Pro against.

System Performance

There’s little doubt that Apple has crafted a great SoC with the A12X, offering an eight core CPU for the first time in an iOS device. Apple claims that the iPad Pro is more powerful than 92% of the available laptops in the market. It does this in a device that is thinner, lighter, and much more efficient than any laptop on the market, which is a testament to their chip design team.

The iPad Pro ships with up to 6 GB of LPDDR4X – 6GB in the 1TB storage SKUs, and 4GB for the rest – compared to 4 GB in the iPhone XS. This is far less than most laptops, which can offer 16-32 GB of RAM with the low-power CPUs. But iOS is certainly less RAM hungry compared to the PC, thanks to the more limited applications available, so RAM is not going to be a limiting factor in most workloads.

Ultimately, despite Apple stating the iPad is quicker than most laptops, it is generally difficult to compare across these platforms because the desktop tools we normally use don’t exist on iOS, and most iOS tools don't exist on desktop OSes. Plus the locked-down nature of the operating system means that even those that do exist generally have lower system access. But we do have some cross-platform tools available.

TabletMark 2017

TabletMark 2017 Overall

TabletMark 2017 Web//Email

TabletMark 2017 Photos/Video

First up is Bapco’s TabletMark 2017, which is their latest iteration of their cross-platform performance and battery life testing tool. The 2017 version has the Windows version written entirely using UWP, and tests two scenarios for performance. Web and Email is the first scenario, and Photo and Video Sharing is the second. Each platform uses its native APIs and tools. Scores are calibrated against the Microsoft Surface 3 tablet, with four Atom cores, and that model is scored to 1000. Systems that are twice as fast would score 2000, and so on. The benchmark is sensitive to both CPU and GPU performance, but the CPU tests are mostly single-threaded.

In this test, the iPad Pro scores below the Surface Pro and Surface Book (which was run as iGPU only) which is perhaps not indicative of the actual performance of the iPad Pro, and once again points to the issues with testing cross-platform, even with companies doing their best to provide as fair of a comparison as possible.

Speedometer 2.0

Speedometer 2.0 - OS WebView

And with that single test out of the way, we’re already into web-based testing, which is important, but doesn’t really give a great look at the underlying hardware due to the scripting engine having such a profound impact on scores.

The new A12X scores slightly higher than the iPhone in this test, but the two extra cores don’t offer a meaningful impact in performance in all scenarios. All of these results are well over the PC though, which struggles in this WebKit created benchmark. Our PC scores are done in the native browser, Edge, but even Chrome on the Surface Book 2 only scores 75.8.

WebXPRT 3

WebXPRT 3 - OS WebView

The latest version of Bapco’s web browser test is WebXPRT, and it offers quite a few different workloads compared to Speedometer. The performance on the PC surpasses the iPad in this test, but once again the underlying scripting engines have a large impact on the performance differences, which is why it is difficult to use these tests as a good cross-comparison.

Kraken 1.1

Mozilla Kraken 1.1

Once again you can see the combination of Apple’s Vortex CPUs combined with their scripting engine in Safari make for a potent combination, scoring well above any of the Windows 10 devices.

System Performance Conclusion

There is little doubt the Apple A12X SoC is potent. Apple claims it is faster than 92% of laptops available on the market, and there isn’t much evidence to refute this, but there really just isn’t a good breadth of evidence at all. A12X on iOS is very fast, and the less complicated applications on iOS are not going to cause this tablet to even break a sweat. A more telling test, perhaps, will be once Adobe has ported over the full-fat version of Photoshop to the iPad, which is expected next year.

Comparing it to the PC though is difficult, since there just are not a lot of good tools available. We will look into getting a proper SPEC comparison in the future which should give us a better baseline. Ultimately comparing iOS performance to the PC is similar to comparing it to Android, and you end up mostly looking at ECMAScript performance on the web.

The SPEC results we do have now though show that the A12X is roughly on-par with the single-threaded performance of the iPhone XS, which isn’t really a surprise, but that the A12X is well ahead of other ARM based CPUs. We’ll need a bit more time to make the same sort of comparison to the PC.

Powering iPad Pro: A12X GPU Performance
POST A COMMENT

135 Comments

View All Comments

  • jeremyshaw - Tuesday, December 04, 2018 - link

    Crikey, that's a fast chip.

    That question about the xbox one s class GPU does raise questions. Why does the Xbox One S draw so much power?
    Reply
  • axfelix - Tuesday, December 04, 2018 - link

    Because it's still using an AMD GPU architecture from 2013, and Apple's and Nvidia's architectures are >3x as powerful per watt at this point. Reply
  • PeachNCream - Tuesday, December 04, 2018 - link

    Eh, NV is just as bad. Their current gen products (GT 1030 aside) generally need more than 75W of power and occupy space equal to two PCI-E slots. Reply
  • Pyrate2142 - Tuesday, December 04, 2018 - link

    Yeah, but those 75W and above cards are operating at a significantly higher performance. You cannot really compare it straight like rust, because 1- not only is the NV cards doing full FP32 compute compared to mixed FP16 and FP32 on the iPad, meaning it is inherently a more strenuous workload to begin with. 2- performance scaling is not a linear function

    In short we can't really take those claims at face value because A- we don't have a way to measure and compare performance in the first place (which brings me to the question of how is apple actually comparing? Using TFLOP performance? Because TFLOP is not an accurate way of measuring GPU performance as a GPU has to do more than just FLOP. Take a RX580 at almost 7 TFLOP and a similar GTX1060 6GB at 4.5TFLOP in FP32. The TFLOP difference suggests a huge performance differences butcher they both perform similarly.) and B- again NV doesn't really make cards that scale down to what the iPad is having. In short, best case it's truly an apples to oranges comparison and I don't think you can directly translate that GPU in the A12X performance against AND or NV because it just not the same comparison both in power target of even how the performance is measured
    Reply
  • Spunjji - Wednesday, December 05, 2018 - link

    Just responding firstly to endorse your comment, and secondly to note that Nvidia do make something at that scale - the 256 CUDA-core Pascal GPU in Tegra X2 would be a solid point of comparison, were it not basically impossible to perform one. Reply
  • olde94 - Wednesday, December 05, 2018 - link

    For power/performance i have a few inputs.

    When looking at Nvidia jetsons running X2 and X1 most performance improvement are on the CPU side of things.

    Also for power refference. The Nvidia shield is not a portable device, and the nintendo switch, running the older version of the 256 cuda core SoC have the GPU running at 764mhz in docked mode and 324 in handheld. The reason is a combination of the battery and the active 30mm fan + somewhat heatsink, cooling solution. The charger is 40W charger, and while it does charge the battery, i will assure you no more than 15W is used for this, and based on charging time during full load, it's less than 10W. Note also that the screen is NOT on.

    An nvidia TX2 is rated at ~20W if i recall, making it WAY more power hungy than the A12 chip
    Reply
  • PeachNCream - Thursday, December 06, 2018 - link

    Eh, the A12X puts a lot into perspective when it comes to compute performance. The big three players in the x86 CPU and GPU space are chasing performance at a cost of rising TDP, at least the phone and tablet competition is highly constrained by power and thermal limits inherent to the platform. The result is that the technological improvements we see in those highly mobile products generally focus on both power and performance. Its a pity to see stupid dual slot coolers on graphics cards to that have to cope with TDPs that range from 75 to an absolutely irrational 200+ watts and processors that blow their TDP budget by 50% under load. I had a Packard Bell 386 PC that was happy with a 60W internal power supply. Computers in 2018 are stupid. They shouldn't even need cooling fans at this point or heatsinks. That old Packard Bell ran a bare IC without even so much as a piece of metal glued atop it and under load, you could rest your thumb on the CPU and it would feel warm, but not hot to the touch. Reply
  • tipoo - Tuesday, December 04, 2018 - link

    It's several fabrication node shrinks back (28nm vs 7nm) and on a 2013 architecture.

    You could probably get something close-ish to XBO performance in a handheld Xbox on 7nm, that would be an interesting product if it had full compatibility...
    Reply
  • axfelix - Tuesday, December 04, 2018 - link

    The Xbox One S (which I think is the comparison here) is actually on 16nm, though it's still that 2013 architecture. I think Apple gets about 2/3 of the advantage from the architecture and 1/3 from the process, and it does work out still to >3x efficiency. Reply
  • tipoo - Tuesday, December 04, 2018 - link

    16nm did substantially cut its power use, and 16nm was less of a node leap than 7 (iirc it was closer to 22nm, but one of the finfett rebrandings?)

    Xbox One S 35-90
    Xbox One 70-120

    http://energyusecalculator.com/electricity_gamecon...
    Reply

Log in

Don't have an account? Sign up now