CPU Web Tests

One of the issues when running web-based tests is the nature of modern browsers to automatically install updates. This means any sustained period of benchmarking will invariably fall foul of the 'it's updated beyond the state of comparison' rule, especially when browsers will update if you give them half a second to think about it. Despite this, we were able to find a series of commands to create an un-updatable version of Chrome 56 for our 2017 test suite. While this means we might not be on the bleeding edge of the latest browser, it makes the scores between CPUs comparable.

All of our benchmark results can also be found in our benchmark engine, Bench.

SunSpider 1.0.2: link

The oldest web-based benchmark in this portion of our test is SunSpider. This is a very basic javascript algorithm tool, and ends up being more a measure of IPC and latency than anything else, with most high-performance CPUs scoring around about the same. The basic test is looped 10 times and the average taken. We run the basic test 4 times.

Web: SunSpider on Chrome 56

Mozilla Kraken 1.1: link

Kraken is another Javascript based benchmark, using the same test harness as SunSpider, but focusing on more stringent real-world use cases and libraries, such as audio processing and image filters. Again, the basic test is looped ten times, and we run the basic test four times.

Web: Mozilla Kraken 1.1 on Chrome 56

Google Octane 2.0: link

Along with Mozilla, as Google is a major browser developer, having peak JS performance is typically a critical asset when comparing against the other OS developers. In the same way that SunSpider is a very early JS benchmark, and Kraken is a bit newer, Octane aims to be more relevant to real workloads, especially in power constrained devices such as smartphones and tablets.

Web: Google Octane 2.0 on Chrome 56

WebXPRT 2015: link

While the previous three benchmarks do calculations in the background and represent a score, WebXPRT is designed to be a better interpretation of visual workloads that a professional user might have, such as browser based applications, graphing, image editing, sort/analysis, scientific analysis and financial tools.

Web: WebXPRT 15 on Chrome 56

Overall, all of our web benchmarks show a similar trend. Very few web frameworks offer multi-threading – the browsers themselves are barely multi-threaded at times – so Threadripper's vast thread count is underutilized. What wins the day on the web are a handful of fast cores with high single-threaded performance.

Benchmarking Performance: CPU Rendering Tests Benchmarking Performance: CPU Encoding Tests
Comments Locked

347 Comments

View All Comments

  • launchcodemexico - Thursday, August 10, 2017 - link

    Why did you end all the gaming review sections with something like "Switching it to Game mode would have made better numbers..."? Why didn't you run the benchmarks in Gaming mode in the first place?
  • Ian Cutress - Thursday, August 10, 2017 - link

    Gaming mode is not default, and we run gaming mode alongside the default - there's two sets of values in each graming test.
  • DanNeely - Thursday, August 10, 2017 - link

    You might want to call that out more clearly in the text. I also missed that you have two sets of 1950X results; and probably wouldn't've figured out what the -G suffix meant without a hint.
  • Ian Cutress - Thursday, August 10, 2017 - link

    I mentioned it in the Game vs Creator mode page, but I'll propagate it through.
  • lordken - Thursday, August 10, 2017 - link

    read before you complain, it is stated at beginning of the review that -G is for game mode...
  • DanNeely - Thursday, August 10, 2017 - link

    Especially during the work day a lot of people just are doing quick glances at the most interesting parts. I'll end to end read it sometime tonight.
  • mapesdhs - Thursday, August 10, 2017 - link

    If people quick-glance, that's their problem for missing key info. :D When learning about something as new as this, I read everything. Otherwise, it's like the tech equivalent of crossing a road while gawping at a phone. :}

    Last time I read so much about a new CPU launch was Nehalem/X58.

    Ian.
  • smilingcrow - Thursday, August 10, 2017 - link

    It seemed really clear to me but for people who didn't read the long text on NUMA etc maybe not.
    The dangers of skimming!
  • mapesdhs - Friday, August 11, 2017 - link

    Indeed. :D Reminds me of when a long time ebay seller told me that long item decriptions are pointless, because most bidders only read the first paragraph, often only the first sentence.
  • Ian Cutress - Thursday, August 10, 2017 - link

    The test suite is a global glove: rather than have 20 tests for each segment, it's a global band of 80 tests for every situation. Johan does different tests as his office is several hundred miles away from where I am (and we're thousands of miles away from any other reviewer).

    For the gaming benchmarks, there are big differences in 99th percentile frame rates and Time Under analysis. As games become more and more GPU bottlenecked for average frame rates, this is where the differentiation point is. It's a reason why we still test 1080p as well. With regards the AI test, I've asked the Civ team repeatedly to make the AI test accessible from the command line so I can rope it into my testing scripts easily (they already do it with the main GPU test). But like many other game studios, getting them to unlock a flag is a frustrating endeavor when they don't even respond to messages.

Log in

Don't have an account? Sign up now