Performance Consistency - Standard Deviation

The second new data set is standard deviation. It gives us a more comprehensive idea of how the average IOPS really varies because in the end, worst-case IOPS is just showing the slowest IO of every second. That does not tell how the rest of the IOs are doing, which is where standard deviation becomes useful.

Once again, the blue dots represent average IOPS, while the red dots are now standard deviation instead of worst-case IOPS. The graphs are dual-axis with IOPS on the left and standard deviation on the right with its own scale. For these graphs, the lower the standard deviation, the less variance there is in performance and the more consistent the performance is, which is better.

Samsung 845DC PRO 400GB

The 845DC EVO and PRO are awesome again. The EVO sports a standard deviation of around one, which is slightly better than what the S3500 offers and significantly better than M500DC's. Given that EVO's IOPS is very comparable with the S3500's, the EVO presents some serious competition to Intel despite the fact that the EVO uses slower TLC NAND with the same 12.7% over-provisioning.

The PRO does even better and is without a doubt the most consistent enterprise SATA SSD. The S3700 comes close, but when you include that fact that the 845DC PRO provides around 15K more IOPS, there is no doubt that it is a better drive for write intensive workloads that require high consistency.

Samsung 845DC PRO 400GB

Samsung 845DC PRO 400GB

Performance Consistency - Worst-Case IOPS Initial Thoughts
Comments Locked

31 Comments

View All Comments

  • Laststop311 - Wednesday, September 3, 2014 - link

    Wish the consumer m2 drives would be released already. Samsung sm951 with pcie gen 3.0 x4 controller would be nice to be able to buy.
  • tuxRoller - Wednesday, September 3, 2014 - link

    All chart titles are the same on page five (performance consistency average iops).
  • tuxRoller - Wednesday, September 3, 2014 - link

    Actually, all the charts carry the same title, but different data.
  • Kristian Vättö - Thursday, September 4, 2014 - link

    The titles are basically "name of the SSD and its capacity - 4KB Random Write (QD32) Performance". The name of the SSD should change when you select a different SSD but every graph has the "4KB Random Write (QD32) Performance" attached to it.
  • CountDown_0 - Wednesday, September 3, 2014 - link

    Hi Kristian,
    a small suggestion: when talking about worst case IOPS you write that "The blue dots in the graphs stand for average IOPS just like before, but the red dots show the worst-case IOPS for every second." Ok, but I'd write it in the graph legend instead.
  • Kristian Vättö - Thursday, September 4, 2014 - link

    It's something I thought about and can certainly consider adding it in the future.
  • rossjudson - Thursday, September 4, 2014 - link

    I'd suggest the following. Use FIO to do your benchmarking. It supports generating and measuring just about every load you'd care about. You can also use it in a distributed mode, so you can run as many tests as you have hardware to support, at the same time.

    Second, don't use logarithmic axes on your charts. The drives you describe here take *huge* dropoffs in performance after their caches fill up and they have to start "working for a living". You are masking this performance drop by not using linear measures.

    Third, divide up your time axis into (say) 60 second chunks, and show the min/max/95/99/99.9/99.9 latency marks. Most enterprise customers care about sustained performance and worst case performance. A really slow IO is going to hold up a bunch of other stuff. There are two ways out of that: Speculative IO (wait a little while for success then issue another IO to another device), or manage and interleave background tasks (defrag/garbage collect) very carefully in the storage device. Better yet, don't have the problem at all. The marketing stats on these drives have nothing to do with the performance they exhibit when they are subject to non-stop, mixed loads.

    Unless you are a vendor that constantly tests precisely those loads, and ensures they work, stay working, and stay tight on latency.
  • SuperVeloce - Thursday, September 4, 2014 - link

    Great review... but dropdown menu for graphs annoys me. ugh
  • Kristian Vättö - Thursday, September 4, 2014 - link

    What do you find annoying in them? I can certainly consider alternative options if you can suggest any.
  • grebic - Thursday, October 2, 2014 - link

    Hi Kristian. I need to bother you with a question: do you think isit worth it to stick this SSD in a NAS? I have a ''fanless'' QNAP HS-210, 2 bay small form NAS, without drives for the moment, so in order to have a complete zero noise and time ''resistence'' to go for SSDs. But I was forgoten what was mentioned here "no wear leveling, no garbage collection'', so I'm wondering if in time the performances will decrease dramatically I'm thinking that the OS of NAS is not knowing to do such ''treatments'' over SSDs for maintaining performances, no? It's not in my intention to do operations over operations on NAS but I would like to know that my data will be ''safe'' and easy ''accesible'' over long time, OK? Very appreciated your oppinion. Thanks, Cristian

Log in

Don't have an account? Sign up now