The Google Nexus 9 Reviewby Joshua Ho & Ryan Smith on February 4, 2015 8:00 AM EST
- Posted in
- Project Denver
- Nexus 9
- Android 5.0
The Secret of Denver: Binary Translation & Code Optimization
As we alluded to earlier, NVIDIA’s decision to forgo a traditional out-of-order design for Denver means that much of Denver’s potential is contained in its software rather than its hardware. The underlying chip itself, though by no means simple, is at its core a very large in-order processor. So it falls to the software stack to make Denver sing.
Accomplishing this task is NVIDIA’s dynamic code optimizer (DCO). The purpose of the DCO is to accomplish two tasks: to translate ARM code to Denver’s native format, and to optimize this code to make it run better on Denver. With no out-of-order hardware on Denver, it is the DCO’s task to find instruction level parallelism within a thread to fill Denver’s many execution units, and to reorder instructions around potential stalls, something that is no simple task.
Starting first with the binary translation aspects of DCO, the binary translator is not used for all code. All code goes through the ARM decoder units at least once before, and only after Denver realizes it has run the same code segments enough times does that code get kicked to the translator. Running code translation and optimization is itself a software task, and as a result this task requires a certain amount of real time, CPU time, and power. This means that it only makes sense to send code out for translation and optimization if it’s recurring, even if taking the ARM decoder path fails to exploit much in the way of Denver’s capabilities.
This sets up some very clear best and worst case scenarios for Denver. In the best case scenario Denver is entirely running code that has already been through the DCO, meaning it’s being fed the best code possible and isn’t having to run suboptimal code from the ARM decoder or spending resources invoking the optimizer. On the other hand then, the worst case scenario for Denver is whenever code doesn’t recur. Non-recurring code means that the optimizer is never getting used because that code is never seen again, and invoking the DCO would be pointless as the benefits of optimizing the code are outweighed by the costs of that optimization.
Assuming that a code segment recurs enough to justify translation, it is then kicked over to the DCO to receive translation and optimization. Because this itself is a software process, the DCO is a critical component due to both the code it generates and the code it itself is built from. The DCO needs to be highly tuned so that Denver isn’t spending more resources than it needs to in order to run the DCO, and it needs to produce highly optimal code for Denver to ensure the chip achieves maximum performance. This becomes a very interesting balancing act for NVIDIA, as a longer examination of code segments could potentially produce even better code, but it would increase the costs of running the DCO.
In the optimization step NVIDIA undertakes a number of actions to improve code performance. This includes out-of-order optimizations such as instruction and load/store reordering, along register renaming. However the DCO also behaves as a traditional compiler would, undertaking actions such as unrolling loops and eliminating redundant/dead code that never gets executed. For NVIDIA this optimization step is the most critical aspect of Denver, as its performance will live and die by the DCO.
Denver's optimization cache: optimized code can call other optimized code for even better performance
Once code leaves the DCO, it is then stored for future use in an area NVIDIA calls the optimization cache. The cache is a 128MB segment of main memory reserved to hold these translated and optimized code segments for future reuse, with Denver banking on its ability to reuse code to achieve its peak performance. The presence of the optimization cache does mean that Denver suffers a slight memory capacity penalty compared to other SoCs, which in the case of the N9 means that 1/16th (6%) of the N9’s memory is reserved for the cache. Meanwhile, also resident here is the DCO code itself, which is shipped and stored as already-optimized code so that it can achieve its full performance right off the bat.
Overall the DCO ends up being interesting for a number of reasons, not the least of which are the tradeoffs are made by its inclusion. The DCO instruction window is larger than any comparable OoOE engine, meaning NVIDIA can look at larger code blocks than hardware OoOE reorder engines and potentially extract even better ILP and other optimizations from the code. On the other hand the DCO can only work on code in advance, denying it the ability to see and work on code in real-time as it’s executing like a hardware out-of-order implementation. In such cases, even with a smaller window to work with a hardware OoOE implementation could produce better results, particularly in avoiding memory stalls.
As Denver lives and dies by its optimizer, it puts NVIDIA in an interesting position once again owing to their GPU heritage. Much of the above is true for GPUs as well as it is Denver, and while it’s by no means a perfect overlap it does mean that NVIDIA comes into this with a great deal of experience in optimizing code for an in-order processor. NVIDIA faces a major uphill battle here – hardware OoOE has proven itself reliable time and time again, especially compared to projects banking on superior compilers – so having that compiler background is incredibly important for NVIDIA.
In the meantime because NVIDIA relies on a software optimizer, Denver’s code optimization routine itself has one last advantage over hardware: upgradability. NVIDIA retains the ability to upgrade the DCO itself, potentially deploying new versions of the DCO farther down the line if improvements are made. In principle a DCO upgrade not a feature you want to find yourself needing to use – ideally Denver’s optimizer would be perfect from the start – but it’s none the less a good feature to have for the imperfect real world.
Case in point, we have encountered a floating point bug in Denver that has been traced back to the DCO, which under exceptional workloads causes Denver to overflow an internal register and trigger an SoC reset. Though this bug doesn’t lead to reliability problems in real world usage, it’s exactly the kind of issue that makes DCO updates valuable for NVIDIA as it gives them an opportunity to fix the bug. However at the same time NVIDIA has yet to take advantage of this opportunity, and as of the latest version of Android for the Nexus 9 it seems that this issue still occurs. So it remains to be seen if BSP updates will include DCO updates to improve performance and remove such bugs.
Post Your CommentPlease log in or sign up to comment.
View All Comments
coburn_c - Wednesday, February 4, 2015 - linkHigh end my rear end. HTC has made high end tablets, HTC makes high end devices, Google makes high end companies make garbage. Seriously. Google devices are done.
rpmrush - Wednesday, February 4, 2015 - linkOn the cover page it has a tag in the top right saying, "HTX Nexus #2". Should be HTC?
cknobman - Wednesday, February 4, 2015 - linkSo regardless of how nice the hardware it Android (and more importantly most of its apps) still have major issues when it comes to tablets.
Why waste time buying this stuff when you can get a Windows 8 tablet?
I get the best of both tablet and PC worlds in one device without being held hostage by lord overseer Apple dictating my every move or Androids crappy support and busted app ecosystem.
milkod2001 - Wednesday, February 4, 2015 - link@cknobman
there's no Windows 8 tablet out there for the same price,with same quality screen and snappiness as this one or Apple tablets. The closest would be Surface3 but it's much more expensive.
If you happen to know about any please post some links please.
rkhighlight - Wednesday, February 4, 2015 - linkBecause a Windows 8 tablet is worse than a tablet when it comes to tablet user experience and worse than a notebook when it comes to notebook experience. Most people like to have two separate devices rather than owning a product that tries to combine everything. For some people this works but the majority prefers two product.
milkod2001 - Wednesday, February 4, 2015 - linknot necessarily if done right. Lenovo Yoga comes probably the closest with its hybrid(tablet +notebook). It need to be much cheaper if it wants to be acquired by majority though.
melgross - Wednesday, February 4, 2015 - linkThe Yoga got terrible reviews everywhere. It's hardly recommended.
Midwayman - Wednesday, February 4, 2015 - linkI don't think that's true at all. Most people would probably prefer to only pay for one device. The issue is one more of execution than concept. If the surface pro 3 were as light, thin, and as good on battery as high end tablets I certainly would have bought one. At a premium even.
Impulses - Wednesday, February 4, 2015 - linkIt that were true MS wouldn't be able to keep Surface on the shelves and the market for x86 Atom hybrids at <$500 would be comprised of more than a handful of OEM & models...
I think for a certain class of individuals it makes all the sense in the world; students, business travelers, etc. For most people however it's just a compromise on both form factors and not really much of a money saver (specially when you factor the upgrade cycle into the equation).
steven75 - Thursday, February 5, 2015 - linkI have a Windows 8.1 tablet (not RT). It's the worst tablet interface I've used yet. A total bomb. And the touch apps and selection are universally awful.
If it didn't have "classic" Windows desktop, I'd have sent it back to the store.