The Google Nexus 9 Review
by Joshua Ho & Ryan Smith on February 4, 2015 8:00 AM EST- Posted in
- Tablets
- HTC
- Project Denver
- Android
- Mobile
- NVIDIA
- Nexus 9
- Lollipop
- Android 5.0
CPU Performance
While Denver’s architecture is something fascinating to study, it’s important to see how well this translates to the real world. Denver on paper is a beast, but in the real world there are a number of factors to consider, not the least of which is the effectiveness of NVIDIA’s DCO. We’ve laid out that Denver’s best and worst case scenarios heavily ride on the DCO, and for NVIDIA to achieve their best-case performance they need to be able to generate and feed Denver with lots and lots of well optimized code. If Denver spends too much time working directly off of ARM code or can’t do a good job optimizing the recurring code it finds then Denver will struggle. Meanwhile other important factors are in play as well, including the benefits and drawbacks of Denver’s two cores versus competing SoC’s quad A15/A57 configurations, and in thermally constrained scenarios Denver’s ability to deliver good performance while keeping its power consumption in check.
In order to test this and general system performance, we turn our suite of benchmarks that include browser performance tests, general system tests, and game-type benchmarks. As Denver relies on code-morphing to enable out of order execution and speculative execution, most of these benchmarks should be able to show ideal performance as loop performance in Denver is basically second to none. While most of these benchmarks are showing their age, they should be usable for valid comparisons until we move to our new test suite.
The Basemark System test seems to contribute quite strongly to how the Nexus 9 performs in the overall subtest. Given that this is a storage performance benchmark, it's likely that Basemark OS II has issues similar to Androbench on 5.0 Lollipop or that random I/O is heavily prioritized in this test.
There's a noticeable performance uplift in the graphics test, and although not exactly part of the CPU this does seem at least somewhat plausible as GPU driver updates can improve performance over time.
Overall, performance seems to be quite checkered, although improved from our initial evaluation of the Nexus 9. Unfortunately, even in benchmarks where the DCO should be able to easily unroll loops to achieve massive amounts of performance, we see inconsistent performance in Denver. This may come down to an issue with the DCO, or even more simply the fact that Denver is spending more time than it would like to directly executing ARM code as opposed to going through the DCO.
In this case looking at the SunSpider and Kraken javascript benchmarks offers an interesting proxy case for exactly that scenario. SunSpider on modern CPUs executes extremely quickly, so quickly that the individual tests are often over in only a couple of dozen of milliseconds. This is a particularly rough scenario for Denver, as it doesn’t provide Denver with much time to optimize, even if the code is run multiple times. Meanwhile Kraken pushes many similar buttons, but its tests are longer, and that gives Denver more time to optimize. Consequently we find that Denver’s SunSpider performance is quite poor – underperforming even the A15-based Tegra K1-32 – while Denver passes even the iPad Air 2 in Kraken.
Ultimately this kind of inconsistent performance is a risk and a challenge for Denver. While no single SoC tops every last CPU benchmark, we also don’t typically see the kind of large variations that are occurring with Denver. If Denver’s lows are too low, then it definitely impacts the suitability of the SoC for high-end devices, as users have come to expect peppy performance at all times.
In practice, I didn't really notice any issues with the Nexus 9's performance, although there were odd moments during intense multitasking where I experienced extended pauses/freezes that were likely due to the DCO getting stuck somewhere in execution, seeing as how the DCO can often have unexpected bugs such as repeated FP64 multiplication causing crashes. In general, I noticed that the device tended to also get hot even on relatively simple tasks, which doesn't bode well for battery life. This is localized to the top of the tablet, which should help with user comfort although this comes at the cost of worse sustained performance.
Facebook
Twitter
RSS
169 Comments
View All Comments
dtgoodwin - Wednesday, February 4, 2015 - link
I really appreciate the depth that this article has, however, I wonder if it would have been better to separate the in depth CPU analysis for a separate article. I will probably never remember to come back to the Nexus 9 review if I want to remember a specific detail about that CPU.nevertell - Wednesday, February 4, 2015 - link
Has nVidia exposed that they would provide a static version of the DCO so that app developers would be able to optimize their binaries at compile time ? Or do these optimizations rely on the program state when they are being executed ? From a pure academic point of view, it would be interesting to see the overhead introduced by the DCO when comparing previously optimized code without the DCO running and running the SoC as was intended.Impulses - Wednesday, February 4, 2015 - link
Nice in depth review as always, came a little late for me (I purchased one to gift it, which I ironically haven't done since the birthday is this month) but didn't really change much as far as my decision so it's all good...I think the last remark nails it, had the price point being just a little lower most of the minor QC issues wouldn't have been blown up...
I don't know if $300 for 16GB was feasible (pretty much the price point of the smaller Shield), but $350 certainly was and Amazon was selling it for that much all thru Nov-Dec which is bizarre since Google never discounted it themselves.
I think they should've just done a single $350-400 32GB SKU, saved themselves a lot of trouble and people would've applauded the move (and probably whined for a 64GB but you can't please everyone). Or a combo deal with the keyboard, which HTC was selling at 50% at one point anyway.
Impulses - Wednesday, February 4, 2015 - link
No keyboard review btw?JoshHo - Thursday, February 5, 2015 - link
We did not receive the keyboard folio for review.treecats - Wednesday, February 4, 2015 - link
Where is the comparison to NEXUS 10????Maybe because Nexus 10's battery life is crap after 1 year of use!!!
Please come back review it again when you used it for a year.
treecats - Wednesday, February 4, 2015 - link
My previously holds true for all the Nexus device line I own.I had Nexus 4,
currently have Nexus 5, and Nexus 10. All the Nexus devices I own have bad battery life after 1 year of use.
Google, fix the battery problem.
blzd - Friday, February 6, 2015 - link
That tells me you are mistreating your batteries. You think it's coincidence that it's happening to all your devices? Do you know how easy it is for batteries to degrade when over heating? Do you know every battery is rated for a certain number of charges only?Mostly you want to avoid heat, especially while charging. Gaming while charging? That's killing the battery. GPS navigation while charging? Again, degrading the battery.
Each time you discharge and charge the battery you are using one of it's charge cycles. So if you use the device a lot and charge it multiple times a day you will notice degradation after a year. This is not unique to Google devices.
grave00 - Sunday, February 8, 2015 - link
I don't think you have the latest info on how battery charging vs battery life works.hstewartanand - Wednesday, February 4, 2015 - link
Even though I personal have 6 tablets ( 2 iPads, 2 Windows 8.1 and 2 android ) and as developer I find them technically inferior to Actual PC - except for Windows 8.1 Surface Pro.I recently purchase an Lenovo y50 with i7 4700 - because I desired AVX 2 video processing. To me ARM based platforms will never replace PC devices for certain applications - like Video processing and 3d graphics work.
I am big fan of Nvidia GPU's but don't care much for ARM cpus - I do like the completion that it given to Intel to produce low power CPU's for this market
What I really like to see is a true technical bench mark that compare the true power of cpus from ARM and Intel and rank them. This includes using extended instructions like AVX 2 on Intel cpus.
Compared this with equivalent configured Nvidia GPU on Intel CPU - and I would say ARM has a very long way to go.
But a lot depends on what you doing with the device. I am currently typing this on a 4+ year old Macbook Air - because it easy to do it and convenient. My other Windows 8.1 ( Lenovo 2 Mix 8 - Intel Adam Baytrail ) has roughly the same speed - but Macbook AIR is more convenient. My primary tablet is the Apple Mini with Retina screen, it is also convent for email and amazon and small stuff.
The problem with some of bench marks - is that they maybe optimized for one platform more than another and dependent on OS components which may very between OS environments. So ideal the tests need to native compile for cpu / gpu combination and take advantage of hardware. I don't believe such a benchmark exists. Probably the best way to do this get developers interested in platforms to come up with contest for best score and have code open source - so no cheating. It would be interesting to see ranking of machines from tablets, phones, laptop and even high performance xeon machines. I also have an 8+ Year old dual Xeon 5160 Nvidia GTX 640 (best I can get on this old machine ) and I would bet it will blow away any of this ARM based tablets. Performance wise it a little less but close to my Lenovo y50 - if not doing VIDEO processing because of AVX 2 is such significant improvement.
In summary it really hard to compare performance of ARM vs Intel machines. But this review had some technical information that brought me back to my older days when writing assembly code on OS - PC-MOS/386