The Google Nexus 9 Review
by Joshua Ho & Ryan Smith on February 4, 2015 8:00 AM EST- Posted in
- Tablets
- HTC
- Project Denver
- Android
- Mobile
- NVIDIA
- Nexus 9
- Lollipop
- Android 5.0
CPU Performance
While Denver’s architecture is something fascinating to study, it’s important to see how well this translates to the real world. Denver on paper is a beast, but in the real world there are a number of factors to consider, not the least of which is the effectiveness of NVIDIA’s DCO. We’ve laid out that Denver’s best and worst case scenarios heavily ride on the DCO, and for NVIDIA to achieve their best-case performance they need to be able to generate and feed Denver with lots and lots of well optimized code. If Denver spends too much time working directly off of ARM code or can’t do a good job optimizing the recurring code it finds then Denver will struggle. Meanwhile other important factors are in play as well, including the benefits and drawbacks of Denver’s two cores versus competing SoC’s quad A15/A57 configurations, and in thermally constrained scenarios Denver’s ability to deliver good performance while keeping its power consumption in check.
In order to test this and general system performance, we turn our suite of benchmarks that include browser performance tests, general system tests, and game-type benchmarks. As Denver relies on code-morphing to enable out of order execution and speculative execution, most of these benchmarks should be able to show ideal performance as loop performance in Denver is basically second to none. While most of these benchmarks are showing their age, they should be usable for valid comparisons until we move to our new test suite.
The Basemark System test seems to contribute quite strongly to how the Nexus 9 performs in the overall subtest. Given that this is a storage performance benchmark, it's likely that Basemark OS II has issues similar to Androbench on 5.0 Lollipop or that random I/O is heavily prioritized in this test.
There's a noticeable performance uplift in the graphics test, and although not exactly part of the CPU this does seem at least somewhat plausible as GPU driver updates can improve performance over time.
Overall, performance seems to be quite checkered, although improved from our initial evaluation of the Nexus 9. Unfortunately, even in benchmarks where the DCO should be able to easily unroll loops to achieve massive amounts of performance, we see inconsistent performance in Denver. This may come down to an issue with the DCO, or even more simply the fact that Denver is spending more time than it would like to directly executing ARM code as opposed to going through the DCO.
In this case looking at the SunSpider and Kraken javascript benchmarks offers an interesting proxy case for exactly that scenario. SunSpider on modern CPUs executes extremely quickly, so quickly that the individual tests are often over in only a couple of dozen of milliseconds. This is a particularly rough scenario for Denver, as it doesn’t provide Denver with much time to optimize, even if the code is run multiple times. Meanwhile Kraken pushes many similar buttons, but its tests are longer, and that gives Denver more time to optimize. Consequently we find that Denver’s SunSpider performance is quite poor – underperforming even the A15-based Tegra K1-32 – while Denver passes even the iPad Air 2 in Kraken.
Ultimately this kind of inconsistent performance is a risk and a challenge for Denver. While no single SoC tops every last CPU benchmark, we also don’t typically see the kind of large variations that are occurring with Denver. If Denver’s lows are too low, then it definitely impacts the suitability of the SoC for high-end devices, as users have come to expect peppy performance at all times.
In practice, I didn't really notice any issues with the Nexus 9's performance, although there were odd moments during intense multitasking where I experienced extended pauses/freezes that were likely due to the DCO getting stuck somewhere in execution, seeing as how the DCO can often have unexpected bugs such as repeated FP64 multiplication causing crashes. In general, I noticed that the device tended to also get hot even on relatively simple tasks, which doesn't bode well for battery life. This is localized to the top of the tablet, which should help with user comfort although this comes at the cost of worse sustained performance.
Facebook
Twitter
RSS
169 Comments
View All Comments
AbRASiON - Thursday, February 5, 2015 - link
LCD, not OLED? Blacks being grey? Nope :/blzd - Friday, February 6, 2015 - link
I'd actually rather grey blacks then the loss of detail in black areas. Pure black is nice, but not when it comes at the expense of shadow details.techn0mage - Thursday, February 5, 2015 - link
I agree that late is better than never. Rather than discuss things that can't be changed, I felt the following points were worth raising:Is there any Nexus 6 data in the benchmark charts? I didn't see any. The N6 and N9 were released roughly around the same point in time, and like the N5 and N7 they are high-profile devices in the Android landscape, so it would have been nice to have them in the charts to make comparisons. Please correct me if I've overlooked anything.
The Denver deep dive, while certainly relevant to Nexus 9 and good AT content on any day, was probably a good candidate for having its own article. I believe it is fair to say the Denver content is -less- time sensitive than the overall review. Hopefully the review was not held back by the decision to include the "DDD" content - and to be clear right now I have no reason to believe it was.
WndlB - Thursday, February 5, 2015 - link
Particularly in this kind of full-dress review of high-end devices, could you start covering the delivered sound, the DAC chips and headphone jack?Via A-B comparisons, I'm finding some real differences and, as people go to more high-quuality audiio streams (plus video sound), this is becoming a differentiator of significance. Thanks.
JoshHo - Tuesday, February 10, 2015 - link
We could do subjective opinion, but properly testing 3.5mm output requires significant investment in test equipment.name99 - Thursday, February 5, 2015 - link
I know this isn't exactly a Nexus9 questions, but how can your battery life results for iPad Air2 be so inconsistent?We are given 10.18 hrs for "display a white image" and 13.63 hrs for "display video". For an OLED display this is possible, but not for a LED-backlit display unless you are running the video at a "base-level" brightness of much lower than the 200 nits of the "display a white image", and what's the point of that? Surely the relevance of the "display a white image" is to show how long the display+battery lasts under normal usage conditions, not when being used as a flashlight?
My point is --- I am guessing that the "display a white image" test utilizes some app that prevents the screen from going black. Do you have confidence that that app (and in particular whatever tickling of the OS that is done to prevent sleep) is doing this in the energy optimal way, on both iOS and Android?
JoshHo - Tuesday, February 10, 2015 - link
I don't believe there was any real background CPU usage. To my knowledge the difference is that Apple enables dynamic contrast in movies.easp - Thursday, February 5, 2015 - link
"The successor to the Nexus 7 was even more incredible, as it pushed hardware that was equal to or better than most tablets on the market at a lower price. However, as with most of these low cost Nexus devices not everything was perfect as corners still had to be cut in order to hit these low price points."So, hardware that was equal or better, except it wasn't? This is a situation where being more specific would help. My guess, when you said equal or better you were referring to certain specifications, certain obvious specifications like core count, RAM, and maybe screen resolution?
mkygod - Friday, February 6, 2015 - link
Owned a Nexus 9 for almost 3 months. I purchased three actually to see if backlight bleed was any better, but nope; so I ended up returning them a couple weeks ago. The bleeding was pretty bad; worse than any LCD device i've ever used and definitely worse than the Nexus 5 and Nexus 7. And it would've been okay if it had uniform bleeding like the Nexus 5, but it had blotches of bright spots all along the edges which is even more distracting. I found the reflectivity with the screen a non-factor in my exclusively indoor use. It's a shame because the Nexus 9 is an otherwise damn good tablet. What's also disappointing, as the review points out, is if you want a high-end tablet around this size, your only options are the 9 and the Tab S. It seems like a lot of really good Android tablets are in the 8" size, such as the Shield and new Dell Venue, with more manufacturers on the horizon making tablets in this size.MartinT - Friday, February 6, 2015 - link
I wonder what level of load penalty is incurred by having to ship in optimized code from main memory. Is there any prefetching going on to preposition code segments in lower level caches ahead of being called?