Testing Notes

Before diving into the nuts and bolts of our actual review, I wanted to stop for a moment and talk about the means and perspectives on how to best test Apple’s rather unparalleled tablet. In terms of mechanical benchmarks, the path is rather straightforward – almost frustratingly so. There are only so many decent standardized benchmarks that run on the iPad Pro, and even fewer of those that overlap with other operating systems, particularly windows. This is why, as atypical and throughput-focused as SPEC is, it remains one of the better tools for determining how the hardware compares to other devices.

However when it comes to the user experience, that’s another matter. The iPad Pro is an iPad, that is also Pro. Specifically, that it has been designed and is being specifically pitched not only as a tool for the iPad’s traditional content-consumption tasks, but also professional productivity use cases as well. Document editing, content creation, photo editing, and other tasks that while not outside of the realm of a regular iPad, aren’t really its forte either.

Since there aren’t really any other serious Arm-based tablets of the iPad Pro’s caliber on the market – Android seems to slip farther and farther behind every year – for the review of this year’s significantly redesigned model, I’m opting to approach matters from the other direction: how does it compare to traditional productivity machines. This is the market occupied by the likes of the Microsoft Surface Pro and other Windows-powered convertibles; devices that have brought the Windows experience to a tablet-like form factor. And while the overlap is by no means perfect, I do feel that these sorts of devices are the standard-bearers for productivity and professionally-oriented tablets on the whole.

So at least in my eyes, the real competition for the new iPad Pros is going to be these other pro devices, rather than a smattering of large-format Arm-based tablets. And these are the sorts of devices I’m primarily going to be looking to compare the iPad Pro against.

System Performance

There’s little doubt that Apple has crafted a great SoC with the A12X, offering an eight core CPU for the first time in an iOS device. Apple claims that the iPad Pro is more powerful than 92% of the available laptops in the market. It does this in a device that is thinner, lighter, and much more efficient than any laptop on the market, which is a testament to their chip design team.

The iPad Pro ships with up to 6 GB of LPDDR4X – 6GB in the 1TB storage SKUs, and 4GB for the rest – compared to 4 GB in the iPhone XS. This is far less than most laptops, which can offer 16-32 GB of RAM with the low-power CPUs. But iOS is certainly less RAM hungry compared to the PC, thanks to the more limited applications available, so RAM is not going to be a limiting factor in most workloads.

Ultimately, despite Apple stating the iPad is quicker than most laptops, it is generally difficult to compare across these platforms because the desktop tools we normally use don’t exist on iOS, and most iOS tools don't exist on desktop OSes. Plus the locked-down nature of the operating system means that even those that do exist generally have lower system access. But we do have some cross-platform tools available.

TabletMark 2017

TabletMark 2017 Overall

TabletMark 2017 Web//Email

TabletMark 2017 Photos/Video

First up is Bapco’s TabletMark 2017, which is their latest iteration of their cross-platform performance and battery life testing tool. The 2017 version has the Windows version written entirely using UWP, and tests two scenarios for performance. Web and Email is the first scenario, and Photo and Video Sharing is the second. Each platform uses its native APIs and tools. Scores are calibrated against the Microsoft Surface 3 tablet, with four Atom cores, and that model is scored to 1000. Systems that are twice as fast would score 2000, and so on. The benchmark is sensitive to both CPU and GPU performance, but the CPU tests are mostly single-threaded.

In this test, the iPad Pro scores below the Surface Pro and Surface Book (which was run as iGPU only) which is perhaps not indicative of the actual performance of the iPad Pro, and once again points to the issues with testing cross-platform, even with companies doing their best to provide as fair of a comparison as possible.

Speedometer 2.0

Speedometer 2.0 - OS WebView

And with that single test out of the way, we’re already into web-based testing, which is important, but doesn’t really give a great look at the underlying hardware due to the scripting engine having such a profound impact on scores.

The new A12X scores slightly higher than the iPhone in this test, but the two extra cores don’t offer a meaningful impact in performance in all scenarios. All of these results are well over the PC though, which struggles in this WebKit created benchmark. Our PC scores are done in the native browser, Edge, but even Chrome on the Surface Book 2 only scores 75.8.


WebXPRT 3 - OS WebView

The latest version of Bapco’s web browser test is WebXPRT, and it offers quite a few different workloads compared to Speedometer. The performance on the PC surpasses the iPad in this test, but once again the underlying scripting engines have a large impact on the performance differences, which is why it is difficult to use these tests as a good cross-comparison.

Kraken 1.1

Mozilla Kraken 1.1

Once again you can see the combination of Apple’s Vortex CPUs combined with their scripting engine in Safari make for a potent combination, scoring well above any of the Windows 10 devices.

System Performance Conclusion

There is little doubt the Apple A12X SoC is potent. Apple claims it is faster than 92% of laptops available on the market, and there isn’t much evidence to refute this, but there really just isn’t a good breadth of evidence at all. A12X on iOS is very fast, and the less complicated applications on iOS are not going to cause this tablet to even break a sweat. A more telling test, perhaps, will be once Adobe has ported over the full-fat version of Photoshop to the iPad, which is expected next year.

Comparing it to the PC though is difficult, since there just are not a lot of good tools available. We will look into getting a proper SPEC comparison in the future which should give us a better baseline. Ultimately comparing iOS performance to the PC is similar to comparing it to Android, and you end up mostly looking at ECMAScript performance on the web.

The SPEC results we do have now though show that the A12X is roughly on-par with the single-threaded performance of the iPhone XS, which isn’t really a surprise, but that the A12X is well ahead of other ARM based CPUs. We’ll need a bit more time to make the same sort of comparison to the PC.

Powering iPad Pro: A12X GPU Performance


View All Comments

  • tipoo - Tuesday, December 4, 2018 - link

    16nm did substantially cut its power use, and 16nm was less of a node leap than 7 (iirc it was closer to 22nm, but one of the finfett rebrandings?)

    Xbox One S 35-90
    Xbox One 70-120

  • PeachNCream - Tuesday, December 4, 2018 - link

    Yup, that is an impressively quick bit of hardware hiding inside the new iPad. It doesn't speak well for a lot of hardware that soaks up a bunch more power and needs a lot more cooling to accomplish similar task. I'm hoping the A12X will be something of a kick in the proverbial pants for the rest of the chip industry to get off their behinds and deliver better performance at much lower TDP, that is both CPU and GPU companies that are inflating real TDP to comparably absurd levels while chasing incremental and insignificant increases in performance. Reply
  • sing_electric - Tuesday, December 4, 2018 - link

    To be fair, the A12 is one of the first 7nm chips to ship. AMD's Ryzen 2700U, in theory, is at least as powerful, but is built on a 12nm process so it consumes more power.

    A year from now, its very likely that AMD will have a mobile Ryzen built on the 7nm same process at the same foundry (TSMC) as Apple, which has the possibility of being more powerful at the same power consumption depending on workload (again, AMD and Apple optimize for different things in their chips, and an.... Apple to oranges comparison is hard to make).

    Within a couple years, we'll see how Intel's new GPU unit does - they've committed to releasing dGPUs but you'd have to think that a side effect would be increased performance of their iGPUs as well. IF they're finally able to start mass-manufacturing on their 10nm (and future) process, they should be competitive as well.
  • Spunjji - Wednesday, December 5, 2018 - link

    +1 to this. Very interested in what Intel can do at low TDPs given their experience there. UHD620 is an old architecture now, so you have to wonder what all the intervening development will grant their first real next-gen GPU.

    AMD have been struggling of late but it sounds like, if they coalesce their development around Navi, they should see some serious benefits at low TDPs too.
  • GruenSein - Tuesday, December 4, 2018 - link

    I actually considered buying the new iPad Pro but as long as iOS and its and 3rd party apps keep working the way they are working, all that performance is wasted IMHO. For the device to be really productive, a real file browser and full access to files is required. Let me organize my files the way, I want. The share button and weird iCloud browser doesn't cut it. Professional workflows require multiple apps to work on the same files, so, as antiquated as Apple wants to make it feel, an "open with" and "save as"-Dialog is crucial. Same goes for file access and network integration. Why can't I access SMB-shares? Most iOS users I know still send files per eMail because that is still the most convenient way to do it. The 3rd party file browsers can help but it is still hardly possible to use these files in any other apps. Some actually start a streaming server if you want to play media with VLC because you cannot tell the app to simply open the file.
    From my point of view, the hardware is great and way ahead of the competition. But the software is keeping it at a toy level for the time being.
  • melgross - Tuesday, December 4, 2018 - link

    Well, far from a toy level, which you would know if you used one. But you’re right that some things are just not available, or not up to snuff. Apple was expected to make major changes to iOS this year for the iPad Pro, but held off until next year due to the rwoerking of the OS for efficiency so that older devices would work better, as well as more modularizing the OS and getting rid of some higher level bugs.

    I hope these expected changes to the Desktop next year not only involve the look and function of the Desktop, but also full use of the USB port for mass storage and hierarchical folders.

    Apple seems to be moving in the right direction, but more slowly than I would like.
  • sing_electric - Tuesday, December 4, 2018 - link

    At this point, I'll believe it when I see it. Adobe had to get very creative with handling files on iOS to bring real Photoshop to the iPad, and they've got the "benefit" of their own cloud platform that they've foist on all their current customers.

    And lack of pointer support is getting harder to stomach - its annoying for pro-users with a keyboard, and it ALSO really limits the ability of people who use assistive communication devices to use iOS AT ALL since a lot of them work via mouse drivers. You'd think adding decent mouse support would be like, a weekend project for a team of engineers.
  • melgross - Tuesday, December 4, 2018 - link

    Adobe ran that file at the presentation while it was residing on the iPad. You can’t do that from the cloud. That’s storage, not live functioning in the app. It doesn’t matter how they did it on the iPad. The fact is that they did, and it was very impressive indeed.

    Pointer support isn’t a matter of a team of engineers. It’s a matter of philosophy. Apple, at this time, still doesn’t believe in it. I don’t happen to agree with their stance. With the original 9.7” iPads, sure, but not for the bigger Pro models. So I agree there.

    But even without it, things work dine in most cases. Would I want to write a novel 9n it? No, but I can get away with several pages of writing.

    iOS is praised for its ability with assisted communications. Not as good as the Mac, but better than Android by a long shot.

    I’m happy to see what happens in the developers conference next June. We should have a good idea where their going then. If nothing much happens, I’d be surprised. I don’t expect everything I want, but some of it.
  • MonkeyPaw - Tuesday, December 4, 2018 - link

    Yeah, mouse support is probably a toggle away in their development process. Anyway, Affinity Photo for iOS is impressive, with both cloud and local storage options. Local photos much be handled through iOS Photos, and traditional file management must be through the files app/cloud service of your choosing. Not as many options as a desktop, but there are at least some options. Reply
  • KPOM - Tuesday, December 4, 2018 - link

    Mouse support is a double-edged sword. It would make some things easier but also make other things harder, particularly if apps “expect” a cursor. Reply

Log in

Don't have an account? Sign up now