Meltdown & Spectre: Analyzing Performance Impacts on Intel's NUC7i7BNH
by Ganesh T S on March 23, 2018 4:15 PM EST- Posted in
- Systems
- Spectre
- Benchmarks
- Meltdown
Concluding Remarks
While the primary purpose of this exercise was just to update our datasets for future system reviews, it none the less proved to be an enlightening one, and something worth sharing. We already had an idea of what to expect going into refreshing our benchmark data for Meltdown and Spectre, and in some ways we still managed to find a surprise or two while looking at Intel's NUC7i7BNH NUC. The table below summarizes the extent of performance loss in various benchmarks.
| Meltdown & Spectre Patches - Impact on the Intel NUC7i7BNH Benchmarks | |
| Benchmark | Performance Notes (Fully Patched vs. Unpatched) |
| BAPCo SYSmark 2014 SE - Overall | -5.47% |
| BAPCo SYSmark 2014 SE - Office | -5.17% |
| BAPCo SYSmark 2014 SE - Media | -4.11% |
| BAPCo SYSmark 2014 SE - Data & Financial Analysis | -2.05% |
| BAPCo SYSmark 2014 SE - Responsiveness | -10.48% |
| Futuremark PCMark 10 Extended | -2.31% |
| Futuremark PCMark 10 Essentials | -6.56% |
| Futuremark PCMark 10 Productivity | -8.03% |
| Futuremark PCMark 10 Gaming | +5.56% |
| Futuremark PCMark 10 Digital Content Creation | -0.33% |
| Futuremark PCMark 8 - Home | -1.9% |
| Futuremark PCMark 8 - Creative | -2.32% |
| Futuremark PCMark 8 - Work | -0.83% |
| Futuremark PCMark 8 - Storage | -1.34% |
| Futuremark PCMark 8 - Storage Bandwidth | -29.15% |
| Futuremark PCMark 7 - PCMark Suite Score | -4.03% |
| Futuremark 3DMark 11- Entry Preset | +2.44% |
| Futuremark 3DMark 13 - Cloud Gate | +1.14% |
| Futuremark 3DMark 13 - Ice Storm | -13.73% |
| Agisoft Photoscan - Stage 1 | -2.09% |
| Agisoft Photoscan - Stage 2 | -12.82% |
| Agisoft Photoscan - Stage 3 | -6.70% |
| Agisoft Photoscan - Stage 4 | -2.84% |
| Agisoft Photoscan - Stage 1 (with GPU) | +1.1% |
| Agisoft Photoscan - Stage 2 (with GPU) | +1.46% |
| Cinebench R15 - Single Threaded | +3.58% |
| Cinebench R15 - Multi-Threaded | -0.32% |
| Cinebench R15 - Open GL | +3.78% |
| x264 v5.0 - Pass I | -1.1% |
| x264 v5.0 - Pass II | -0.75% |
| 7z - Compression | -0.16% |
| 7z - Decompression | -0.38% |
Looking at the NUC – and really this should be on the mark for most SSD-equipped Haswell+ systems – there isn't a significant universal trend. The standard for system tests such as these is +/- 3% performance variability, which covers a good chunk of the sub-benchmarks. What's left then are more meaningful performance impacts in select workloads of the BAPCo SYSmark 2014 SE and Futuremark PCMark 10 benchmarks, particularly storage-centric benchmarks. Other than those, we see certain compute workloads (such as the 2nd stage of the Agisoft Photoscan benchmark) experience a loss in performance of more than 10%.
On the whole, we see that the patches for Meltdown and Spectre affect real-world application benchmarks, but, synthetic ones are largely unaffected. The common factor among most of these benchmarks in turn is storage and I/O; the greater the number of operations, the more likely a program will feel the impact of the patches. Conversely, a compute-intensive workload that does little in the way of I/O is more or less unfazed by the changes. Though there is a certain irony to the fact that taken to its logical conclusion, patching a CPU instead renders storage performance slower, with the most impacted systems having the fastest storage.
As for what this means for future system reviews, the studies done as part of this article give us a way forward without completely invalidating all the benchmarks that we have processed in the last few years. While we can't reevaluate every last system – and so old data will need to stick around for a while longer still – these results mean that the data from unimpacted benchmarks is still valid and relevant even after the release of the Meltdown and Spectre patches. To be sure, we will be marking these results with an asterisk to denote this, but ultimately this will allow us to continue comparing new systems to older systems in at least a subset of our traditional benchmarks. Which combined with back-filling benchmarks for those older systems that we do have, lets us retain a good degree of review and benchmark continuity going forward.
Facebook
Twitter
RSS
83 Comments
View All Comments
akula2 - Sunday, March 25, 2018 - link
Really, are you stupid or what?Don't you know what is happening, Snowden and Wikileaks? You've the audacity to call me as a conspiracy theorist, you moron where were you when Dual_EC_DRBG "vulnerability" was discovered? Who planned that backdoor? Don't you know the collusion of companies behind the epic data collection programme launched by the NSA? Do you really think people on this planet are idiots?
boeush - Monday, March 26, 2018 - link
Yes, Asia - that global bastion of freedom, democracy, open-source transparency, and total absence of corruption. From China, to Vietnam, to North and South Korea, to India and Pakistan, to Myanmar and Indonesia, Afghanistan, Iran, the Philippines, Japan, oh my... LMAOMatthmaroo - Saturday, March 24, 2018 - link
Someone forgot some meds todayTrevorH - Saturday, March 24, 2018 - link
I would love to see some linux testing added to this set up. I work for a VoIP provider and run CentOS on a bunch of servers and the results I see by adding the initial RH patches were an approximate 30% increase in cpu time. Adding the microcode patch and enabling the IBRS mitigation to that resulted in a 100% increase in cpu usage for our workload. Yes, 100% increase - so a machine with 20 cores that was running at 800% cpu usage before the patches was using 1600% (16 cores at 100% each) after both the PTI and IBRS mitigations were turned on. Now our workload is probably quite unusual in that it uses both KVM virtualization and does lots and lots of small packet UDP network i/o but it does mean that with the mitigations in place, in order to run the same workload that we did before, we'd need to buy just about double the hardware we currently have in use.timecop1818 - Sunday, March 25, 2018 - link
Hey but why the fuck did you even install the patches, or enable these "fixes"? You've just said it, you are running a closed voip routing system. Why do you need to care about either of these non-problems in those servers? Why do people running Windows on desktop in single user setup need to care about any of this? In an earlier comment Ryan mentioned there's registry settings to disable this, guess what I'm doing as soon as I'm home?Alexvrb - Saturday, March 24, 2018 - link
You need to test with older architectures. Pre-Broadwellkn0w1 - Sunday, March 25, 2018 - link
Here is one limited comparison for Ivy Bridge and Y-Series Broadwell for good measure.https://www.smajumdar.com/2018/03/musing-48-impact...
HStewart - Saturday, March 24, 2018 - link
Well first all - i not sure if the average customer will even notice these changes.Here is two things I thinking of
1. Has there been actual virus / attack with this stuff - or all of this hypothetical
2. It odd that power consumptions actually improved with the path
The good news this stuff is final over with - move on.
satai - Sunday, March 25, 2018 - link
"The good news this stuff is final over with."Probably not. We can expect more types of Spectre and similar attacks to come...
29a - Saturday, March 24, 2018 - link
I have to give props to ASRock for releasing a new BIOS for my Z170 Extreme3 motherboard dated 2018/3/12, I wasn't sure if they would update a budget Z170 board. This along with every ASRock MB I have owned being super stable has made me a very loyal customer.