One of the bigger consortiums for developing benchmarks is BAPCo. Over the years the company has released a variety of benchmarking tools, such as MobileMark, TabletMark, and SYSmark. This latter software tool is run by a number of high profile OEMs to categorize their systems, and used by numerous government agencies to gauge performance on systems with key metrics. Today BAPCo is announcing its next generation of SYSmark, known as SYSmark 2018.

The BAPCo consortium is a non-profit consortium, with members including Dell, HP, Hitachi, Intel, Lenovo, Microsoft, Pegatron, Samsung, Toshiba, WD, Winstron, and others. The goal of the consortium is to develop objective performance benchmarks, using high profile and common tools as the baseline for the metrics. By virtue of being a high-profile non-profit, and working at government level procurement, BAPCo has had significant success in its benchmark deployments.


Sysmark 2018 Interface, Ryzen 7 2700X Result

Software such as SYSmark 2018 takes a number of software utilities, such as Photoshop, Office, Chrome, and PowerDirector to profile a system with four metrics: an overall score, a productivity score, a creativity score, and a responsiveness score. Recent updates in the last edition enabled power measurement during benchmarks using a Watts Up! meter, which also comes through in the 2018 edition. The new edition has the following tests (taken from the manual):

1. Productivity

The Productivity scenario models productivity usage including word processing (mail merge, document comparison, and PDF conversion), spreadsheet data manipulation (data modeling, financial forecasting), and email creation/management, presentation editing, software development (compiling code), application installation, and archiving files.

  • Adobe® Acrobat® Pro DC
  • AutoIT 3.3.14.2
  • BowPad64-2.3.3 installer
  • Google Chrome™
  • Microsoft® Excel® 2016
  • Microsoft® OneNote® 2016
  • Microsoft® Outlook® 2016
  • Microsoft® PowerPoint® 2016
  • Microsoft® Word® 2016
  • Windows Zip

2. Creativity

The Creativity scenario models editing digital photos (creating HDR and panoramic photos), cataloguing digital photos (organizing catalog, use of facial detection to group people), and editing digital video (create a timeline from various source clips and transcode the output).

  • Adobe® Photoshop® CC 2017
  • Adobe® Lightroom® Classic CC
  • CyberLink PowerDirector® 2015

3. Responsiveness

The Responsiveness scenario includes activities such as multiple application launches, web browsing with many tabs open, and task switching.

  • Adobe® Acrobat® Pro DC
  • Adobe® Lightroom® Classic CC
  • Adobe® Photoshop® CC 2017
  • CyberLink PowerDirector® 2015
  • Google Chrome™
  • Microsoft® Excel® 2016
  • Microsoft® OneNote® 2016
  • Microsoft® Outlook® 2016
  • Microsoft® PowerPoint® 2016
  • Microsoft® Word® 2016

Standard System Comparison

These benchmarks blur the lines between synthetic and real-world, though often land on the side of real-world. By condensing the results into a single number, it can offer an easy way to reflect on the scale of performance, although it also obfuscates some of the nuance.  The results are compared to a base system score of 1000 points, which for this generation will be a low-end Dell Optiplex 5050 Tower:

  • Processor: Intel® Core™ i3-7100 @ 3.90 GHz
  • Operating System: Microsoft Windows 10 Pro x64 (build 1709)
  • RAM: 1x4GB DDR3
  • Graphics: Intel® HD Graphics 630
  • Audio: Integrated High Definition Audio
  • Storage: 128GB capacity, M.2 SATA
  • Networking: Integrated Gigabit Ethernet

This is a step up from SYSmark 2014 1.5, which used an i3-6100 and a spinning hard-drive. With this default system, the i3-7100 has a nice and fast single core performance, is a dua core with hyperthreading, but will be let down by the single channel memory.

New features for SYSmark 2018 include the ability to uninstall the software, run the tests from the command line, and also enable post-processing of sub-test data. All three of which should help reviewers and testers automate into our scripts.

We ran the benchmark on a number of test systems we have set up, and scored the following:

SYSmark 2018: OverallSYSmark 2018: ProductivitySYSmark 2018: CreativitySYSmark 2018: Responsiveness

BAPCo’s benchmark suites are approved by the consortium members before being released. It must be pointed out that AMD left the consortium after SYSmark 2012 was launched (NVIDIA and VIA also left). This came about due to disagreements about the nature and weighting of the workloads that seemed beneficial to the competition. Despite this, SYSmark is still considered by many as a major benchmark for various markets, and since AMD’s newest high performance x86 processor came to market, results in the last generation SYSmark 2014 SE were competitive.

Related Reading

Source: BAPCo

POST A COMMENT

7 Comments

View All Comments

  • Zizy - Friday, August 17, 2018 - link

    I don't understand this i3 7100 @ 4.2 part, nor the quadcore part.
    i3 7100 is a 2C/4T @ 3.9, doesn't have turbo boost and cannot be overclocked.

    Perhaps the i3 7350K is meant here, that one is a 4.2 GHz part (but it still has 2C/4T). There is no 4C i3 at 4.2 GHz unless you OC the 8350K.
    Reply
  • Ian Cutress - Saturday, August 18, 2018 - link

    That was me blindly copying from their testing procedure guide, it didn't click at the time. I've reached out for clarification. Reply
  • patel21 - Friday, August 17, 2018 - link

    Wow, the i3 comes very near to the Ryzen! I would like to see the 8th gen U series i5 and i7 scores too. Reply
  • sbrown23 - Friday, August 17, 2018 - link

    This benchmark clearly doesn't stress a high number of threads in the Productivity and Responsiveness categories. The fact that a 2-core i3 approaches an 8-core Ryzen and $1000, 10-core i7 should tell you something. It makes sense that a lot of those workloads aren't going to push high core count processors to their limit. The average office worker doesn't need 10C/20T or 8C/16T. Reply
  • Ian Cutress - Saturday, August 18, 2018 - link

    Pretty much this. Creativity is multi-threaded, so that's the only time it gets to stretch some legs. Reply
  • demonsavatar - Sunday, August 19, 2018 - link

    Still remember the FTC settlement and the disclaimer Intel had to post that Sysmark was only optimized for Intel parts (which is why AMD, NVIDIA, and VIA all left BAPCO)...has that changed? Do they still need to post that disclaimer?

    It would be interesting to see a ranking of machines using Sysmark 2018 and PCMark10 or something similar. We really need an open-source competitor....
    Reply
  • Gracie1 - Tuesday, September 18, 2018 - link

    Condensing the effects right into a unmarried wide variety it is able to provide an smooth way to mirror on the size of overall performance. The productiveness scenario models productiveness utilization along with word processing mail merge file contrast. The intention of the consortium is to broaden objective overall performance benchmarks using high profile and not unusual tools as the baseline for the metrics. https://www.courseworkcamp.co.uk/ Reply

Log in

Don't have an account? Sign up now