BAPCo SYSmark 2014 SE

BAPCo's SYSmark 2014 SE is an application-based benchmark that uses real-world applications to replay usage patterns of business users in the areas of office productivity, media creation and data/financial analysis. In addition, it also addresses the responsiveness aspect which deals with user experience as related to application and file launches, multi-tasking etc. Scores are calibrated against a reference system that is defined to score 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.

SYSmark scores are based on total application response time as seen by the user, including not only storage latency but time spent by the processor. This means there's a limit to how much a storage improvement could possibly increase scores, because the SSD is only in use for a small fraction of the total test duration. This is a significant difference from our ATSB tests where only the storage portion of the workload is replicated and disk idle times are cut short to a maximum of 25ms.

AnandTech SYSmark SSD Testbed
CPU Intel Core i5-7400
Motherboard ASUS B250-PLUS
Chipset Intel B250
Memory 2x 8GB Kingston DDR4-2400 CL17
Case In Win C583
Power Supply Cooler Master G550M
OS Windows 10 64-bit, version 1703

Our SSD testing with SYSmark uses a different test system than the rest of our SSD tests. This machine is set up to measure total system power consumption rather than just the drive's power.

BAPCo SYSmark 2014 SE - ResponsivenessBAPCo SYSmark 2014 SE - Overall Rating

SYSmark 2014 SE doesn't come close to stressing the storage system enough to show meaningful distinctions between high-end NVMe SSDs. The Responsiveness sub-test is the most sensitive to storage performance, and only shows a few percent difference between most SSDs. The overall test barely registers a difference at all. But with my primary power meter broken, SYSmark does provide a rough assessment of how power hungry the Intel Optane SSD 900P is:

BAPCo SYSmark 2014 SE - Total System Power

The SYSmark energy usage scores measure total system power consumption, excluding the display. Our SYSmark test system idles at around 26 W and peaks at over 60 W measured at the wall during the benchmark run. SATA SSDs seldom exceed 5 W and idle at a fraction of a watt, and the SSDs spend most of the test idle.

Despite being the fastest drive in this bunch and thus the one with the shortest SYSmark run times, the Optane SSD 900P used far more energy over the course of the three SYSmark runs. With higher idle power than even the Intel SSD 750 and similar power draw under load, the Optane SSD 900P really needs its heatsink. Intel is a very long way off from being able to package this level of performance in a M.2 SSD.

Mixed Read/Write Performance Conclusion
Comments Locked

205 Comments

View All Comments

  • ddriver - Friday, October 27, 2017 - link

    I got myself a bucked of salt. The necessary requirement to swallow that Houdini "2.7x better" claim from the launch PR.

    I've been rendering stuff since the days of 3d max for frigging DOS. And I am yet to experience a scenario where CPU load is not in the 99% range.

    Having a rendering job that cannot feed the CPU to above 10% load with the insanely fast 960 pro has got to be an unprecedented case of cooked-up benchmark in human history.
  • extide - Friday, October 27, 2017 - link

    Did you read the article? It pretty clearly explains how they got that result, and it makes sense.
  • ddriver - Friday, October 27, 2017 - link

    Oh yeah, I get it. Hypetane is a synthetic beast. Which allows to showcase said advantage as long as you focus on it in a carefully devised and completely detached from real-world usage workload.

    Don't get me wrong. It is good that hypetane is now available in capacities that actually allow to use it. And if endurance turns out to be tangibly better than nand, I might actually buy it. Low queue depth performance is good, especially random read, which may not be of that much practical use to most of the people out there, but I could make good use of that.

    But it will remain "hypetane" even after I go and buy it. Because intel said "1000 times better", and it is not even 10 times better. A zero on its own might be nothing, but two zeroes after a positive number make quite a lot of difference.
  • ddriver - Friday, October 27, 2017 - link

    "no other alternative nonvolatile memory technology is close to being ready to challenge 3D XPoint"

    Except for SLC, which was so good it was immediately abandoned once inferior and more profit friendly NAND implementations were available.

    A SLC based product coupled with MRAM cache will easily humiliate hypetane in its few strong aspects.

    Too bad NAND drives are now moving to TLC and QLC, even MLC is heading in the "luxury item" category. Too bad because 3D SLC has tremendous potential. Let's see if it gets realized.
  • extide - Friday, October 27, 2017 - link

    How would that work. SLC is slower than Optane, can't be written at a block level, needs trash collection, etc. Then you cache it with a technology similar to Optane? Why not just build a drive with all MRAM, oh yeah, too expensive. Looks like Optane wins.
  • ddriver - Friday, October 27, 2017 - link

    Nope, SLC is actually faster. Look it up.

    And what it cannot do is write at the bit level. Which is not really a big deal. Even CPUs cannot address RAM at bellow a byte, if you want single bit operations, you have to use bitwise operators. Writing at a higher level is actually very efficient, because it reduces overhead. If single bit addressing was important, that's who computers would work.

    Furthermore, single bit writes produce a significant challenge when tracking wear levels. Hypetane still wears out, you know... It will be tremendously harder to accurately track wear at bit level, and I am abot 99.999999% sure it is not how intel does it, meaning that a lot of that supposed extra endurance will be forfeited by managing wear at a coarsely grained level. They won't be managing that at bit level, the overhead will be tremendous and will completely diminish potential advantages.

    The MRAM cache will reduce a lot of write amplification and garbage collection.

    It also looks like 3d SLC has about 3 times the density of the chips intel is currently using for hypetane.

    "Why not just build a drive with all MRAM" - density is too low. Which is also why we use RAM for working memory, I mean volatility can easily be solved by say adding a RTG battery to a DRAM drive, giving it effectively about a century of continuous, uninterrupted power. It is doable, but then again, redundant, and while it is true that the industry does a lot of pointless things nowadays, the only ones that qualify are those with a desirable usability to profitability ratio, and a RTG DRAM drive is simply too good to offer...

    "Looks like Optane wins" - anyone can win when running unopposed. The moment someone makes a SLC/MRAM hybrid and it loses to hypetane, I will retract my statement and admit I was wrong. I have zero problem with that ;)
  • vanilla_gorilla - Friday, October 27, 2017 - link

    So you're saying Optane sucks because it would be slower than a drive that doesn't exist?
  • ddriver - Friday, October 27, 2017 - link

    No, I am saying it "sucks" because for all intents and purposes, it is not any faster than a 2 year old drive that it was supposed to beat by a 1000 times.

    And the reason I put it "sucks" is because I never said it does suck. I give it a very realistic valuation. What sucks is how far that realistic valuation is from what intel promised. Which is entirely on them.
  • name99 - Friday, October 27, 2017 - link

    He's saying two distinct things.
    (a) This costs too much for what it delivers. IF Samsung wanted to compete with it, they could do so with a suite of existing technologies. But they probably won't do so because there is little demand for a product like this; honestly it only exists so that Intel can say "see, 3D-XPoint is too, real".

    (b) The place where 3D-XPoint ACTUALLY makes sense is, more or less, what AnandTech says --- as a slower (but much larger) RAM replacement. That's what plays to the technology's strengths (simple controller, byte-level access). But Intel STILL are not shipping that --- which makes one wonder WTF not?

    It IS reasonable to point out that Intel has been lying about this product since the day it was announced, and that the only reason they're shipping these SSD drives is to throw up more smoke to hide the fact that the actually sensible use case remains (for some reason) impossible.

    Being a fanboy isn't about always praising your company, it's about refusing to criticize your company even when they're clearly in the wrong. Intel is clearly in the wrong here, in the sense that nothing that they promised about Optane is actually reality even today, two years after the announcement.
    If you think that's reasonable behavior, ask yourself how you would react if your favorite villainous company did the same.
    Would you be impressed if AMD announced that they're going to ship a GPU 1000x faster than the competition, and two years later all they have is something 2.7x as fast (under very specialized circumstances)?
    Would you let Apple off the hook if they said that the Apple car was going to have 1000x the range of a Tesla, then they shipped two years later, a car with 2.7x the range of a Tesla?
  • Drumsticks - Saturday, October 28, 2017 - link

    Re: AMD example: if AMD claimed a product would be 100x or 1000x faster than Nvidia, but only delivered something 6-10x faster in the majority of cases, and on par in the rest, for only 2-3x more money, I'd still be pretty satisfied.

Log in

Don't have an account? Sign up now