One of the consequences of the Meltdown and Spectre vulnerabilities, besides the immediate security concerns, is what it has meant for our ongoing testing and benchmarking operations. Along with accounting for the immediate performance changes from the required patches, we've also needed to look at how we present data and how we can (or can't) compare new patched systems to older unpatched systems. From an editorial perspective we need to ensure the continuity of our data as well as providing meaningful benchmark collections.

What we've found so far is that the impact of the Meltdown and Spectre patches varies with the workload and the type of test. Nate's GPU testing shrugs it off almost entirely, while Billy's SSD testing especially feels the pinch. In the middle of course are consumer system reviews, where workloads are more varied but also much more realistic and often more relevant as well. Of course, this is also the type hardware that we most often have to return when we're done, making it the most challenging to correct.

As everyone at AnandTech has been integrating the patches and updating their datasets in their own way, I wanted to offer some insight into what's going on for system testing. With several important systems set to launch in the first half of this year – particularly Intel's Hades Canyon – I've been going back and collecting updated results for use in those future reviews. In the process, I also wanted to document how performance has been impacted by these security patches and which benchmarks in particular have been affected.

Evaluation Setup

The Intel Kaby Lake NUC (NUC7i7BNH) was used for all the benchmarks presented in this article. It was chosen for two reasons. Being a relatively new system that consumers can still purchase in the market, evaluating it presents data of particular relevance to readers. Intel was prompt in providing BIOS updates to resolve the Meltdown and Spectre issues compared to other OEMs whose systems I had in hand for evaluation. The NUC was configured with the following specifications:

Intel NUC7i7BNH (Unpatched) Specifications
Processor Intel Core i7-7567U
Kaby Lake-U, 2C/4T, 3.5 - 4.0 GHz, 14nm+, 4 MB L2, 28W TDP
Memory Crucial Ballistix Sport LT BLS16G4S240FSD.16FAD DDR4
15-15-15-35 @ 2133 MHz
2x16 GB
Graphics Intel Iris Plus Graphics 650
Disk Drive(s) ADATA SX8000NP
(128 GB; M.2 Type 2280 PCIe 3.0 x4 NVMe; Micron 32L 3D MLC)
Networking Intel Dual Band Wireless-AC 8260
(2x2 802.11ac - 866 Mbps)
1x Intel 10/100/1000 RJ-45 GbE
Audio 3.5mm Headphone Jack
Capable of 5.1/7.1 digital output with HD audio bitstreaming (HDMI)
Miscellaneous I/O Ports 4x USB 3.0 Type-A
1x Thunderbolt 3 USB-C
1x micro-SDXC
Operating System Retail unit is barebones, but we installed Windows 10 Enterprise x64
Pricing (Barebones) USD 445
Full Specifications Intel NUC7i7BNH Specifications

Three different patch configurations were benchmarked:

  • Completely unpatched
  • Only OS patches activated (Graph entry : Patch OS)
  • Completely patched with updates to both the OS and the CPU microcode (Graph entry : Patch OS + uCode)

While it will (eventually) be an uncommon configuration, right now a lot of systems are still only running OS patches without their associated microcode updates, as the latter is still being distributed. At the same time, because the microcode patches are primarily for Spectre – Meltdown fixes are implemented at an OS level – this allows us to see just how much of an impact the Spectre fixes have on top of the existing Meltdown fixes.

The completely unpatched system used Windows 10 version 16299.125 with BIOS v0054. As evident from the screenshot below, the system is susceptible to both Spectre and Meltdown.

The second configuration was processed with Windows 10 version 16299.309 with BIOS v0060. The screenshot below shows that the system is protected against Meltdown, but, not Spectre.

The final fully patched configuration was process with Windows 10 version 16299.214 and BIOS v0061. The current BIOS is v0062 (the one we used was available briefly before it was pulled to undergo additional QA for possible random restarts). This configuration shows full protection against both Meltdown and Spectre.

We used our standard test suite for low power desktops / industrial PCs in order to evaluate the impact of the patching on the performance of the system.

BAPCo and Futuremark Benchmarks
POST A COMMENT

84 Comments

View All Comments

  • akula2 - Sunday, March 25, 2018 - link

    Really, are you stupid or what?

    Don't you know what is happening, Snowden and Wikileaks? You've the audacity to call me as a conspiracy theorist, you moron where were you when Dual_EC_DRBG "vulnerability" was discovered? Who planned that backdoor? Don't you know the collusion of companies behind the epic data collection programme launched by the NSA? Do you really think people on this planet are idiots?
    Reply
  • boeush - Monday, March 26, 2018 - link

    Yes, Asia - that global bastion of freedom, democracy, open-source transparency, and total absence of corruption. From China, to Vietnam, to North and South Korea, to India and Pakistan, to Myanmar and Indonesia, Afghanistan, Iran, the Philippines, Japan, oh my... LMAO Reply
  • Matthmaroo - Saturday, March 24, 2018 - link

    Someone forgot some meds today Reply
  • TrevorH - Saturday, March 24, 2018 - link

    I would love to see some linux testing added to this set up. I work for a VoIP provider and run CentOS on a bunch of servers and the results I see by adding the initial RH patches were an approximate 30% increase in cpu time. Adding the microcode patch and enabling the IBRS mitigation to that resulted in a 100% increase in cpu usage for our workload. Yes, 100% increase - so a machine with 20 cores that was running at 800% cpu usage before the patches was using 1600% (16 cores at 100% each) after both the PTI and IBRS mitigations were turned on. Now our workload is probably quite unusual in that it uses both KVM virtualization and does lots and lots of small packet UDP network i/o but it does mean that with the mitigations in place, in order to run the same workload that we did before, we'd need to buy just about double the hardware we currently have in use. Reply
  • timecop1818 - Sunday, March 25, 2018 - link

    Hey but why the fuck did you even install the patches, or enable these "fixes"? You've just said it, you are running a closed voip routing system. Why do you need to care about either of these non-problems in those servers? Why do people running Windows on desktop in single user setup need to care about any of this? In an earlier comment Ryan mentioned there's registry settings to disable this, guess what I'm doing as soon as I'm home? Reply
  • npz - Monday, March 26, 2018 - link

    > Why do people running Windows on desktop in single user setup need to care about any of this?

    That's because a web exploit can be used to obtain information about other processes (like passwords) on your system.
    Reply
  • Alexvrb - Saturday, March 24, 2018 - link

    You need to test with older architectures. Pre-Broadwell Reply
  • kn0w1 - Sunday, March 25, 2018 - link

    Here is one limited comparison for Ivy Bridge and Y-Series Broadwell for good measure.
    https://www.smajumdar.com/2018/03/musing-48-impact...
    Reply
  • HStewart - Saturday, March 24, 2018 - link

    Well first all - i not sure if the average customer will even notice these changes.

    Here is two things I thinking of

    1. Has there been actual virus / attack with this stuff - or all of this hypothetical
    2. It odd that power consumptions actually improved with the path

    The good news this stuff is final over with - move on.
    Reply
  • satai - Sunday, March 25, 2018 - link

    "The good news this stuff is final over with."

    Probably not. We can expect more types of Spectre and similar attacks to come...
    Reply

Log in

Don't have an account? Sign up now