One of the expanding elements of the storage business is that the capacity per drive has been ever increasing. Spinning hard-disk drives are approaching 20 TB soon, while solid state storage can vary from 4TB to 16TB or even more, if you’re willing to entertain an exotic implementation. Today at the Data Centre World conference in London, I was quite surprised to hear that due to managed risk, we’re unlikely to see much demand for drives over 16TB.

Speaking with a few individuals at the show about expanding capacities, storage customers that need high density are starting to discuss maximum drive size requirements based on their implementation needs. One message starting to come through is that storage deployments are looking at managing risk with drive size – sure, a large capacity drive allows for high-density, but in a drive failure of a large drive means a lot of data is going to be lost.

If we consider how data is used in the datacentre, there are several levels regarding how often the data is used. Long-term storage, known as cold storage, is accessed very infrequently and occupied with mechanical hard-drives with long-time data retention. A large drive failure at this level might lose substantial archival data, or require long build times. More regularly accessed storage, or nearline storage / warm storage, is accessed frequently but is often used as a localised cache from the long-term storage. For this case, imagine Netflix storing a good amount of its back-catalogue for users to access – a loss of a drive here requires accessing colder storage, and the rebuild times come in to play. For hot storage, the storage that has constant read/write access, we’re often dealing with DRAM or large database work with many operations per second. This is where a drive failure and rebuild can result in critical issues with server uptime and availability.

Ultimately the size of the drive and the failure rate leads to element of risks and downtime, and aside from engineering more reliant drives, the other variable for risk management is drive size. 16TB, based on the conversations I’ve had today, seems to be that inflection point; no-one wants to lose 16TB of data in one go, regardless of how often it is accessed, or how well a storage array has additional failover metrics.

I was told that sure, drives above 16TB do exist in the market, however aside from niche applications (such as risk is an acceptable factor for higher density), volumes are low. This inflection point, one would imagine, is subject to change based on how the nature of data and data analytics will change over time. Samsung’s PM983 NF1 drive tops out at 16 TB, and while Intel has shown samples of 8 TB units of its long ruler E1.L form factor, it has listed future drives using QLC up to 32TB. Of course, 16 TB per drive puts no limits on the number of drives per system – we have seen 1U units with 36 of these drives in the past, and Intel has been promoting up to 1 PB in a 1U form factor. It is worth noting that the market for 8 TB SATA SSDs is relatively small - no-one wants to rebuild that large a drive at 500 MB/s, which would take a minimum of 4.44 hours, bringing server uptime down to 99.95% rather than the 99.999% metric (5m22 per year).

Related Reading

POST A COMMENT

85 Comments

View All Comments

  • ksec - Saturday, March 16, 2019 - link

    This a reply to my future self in case I need to look things up again. Ian made a reply in Reddit ( why not here ) about the Data being too much to lose and risk were too large for Cold, Warm or even Hot Data.

    I think we fundamentally agree, except everyone were thinking of different use case. Those vendor were likely selling to customer hosting within dozen of PB data. That is anywhere between 1.6% to 0.16% of their Data. Which is quite a lot. But I was thinking in the case of Facebook, Google And Amazon Scale. 1EB, 1000PB, that is 0.0016% of their data. I doubt they would have thought 32TB would be too much, likely at their scale they would even want 100TB.
    Reply
  • thingreenveil313 - Wednesday, March 13, 2019 - link

    How does a live rebuild on an array with hot storage increase downtime? This doesn't make any sense. No one with sense takes down a prod device to do a rebuild when the point of drive pools is to keep devices online through disk failures. Reply
  • ksec - Thursday, March 14, 2019 - link

    Exactly. I would have yawned if it was coming from any other media, but coming from Anandtech I suspect there are uses cases that I don't know or understand. I wish someone could chime in on this. Reply
  • boozed - Wednesday, March 13, 2019 - link

    "if you’re willing to entertain an exotic implementation"
    And an exotic price!
    Reply
  • farmergann - Wednesday, March 13, 2019 - link

    Prices and IO bottlenecks of larger drives are the sole limiting factors to greater adoption.

    Drive failure has basically nothing to do with this conversation and density means squat diddly given the minuscule scaling achieved.

    I can't even bring myself to address the uptime silliness mentioned by this author. Jimeny, please edit that away...
    Reply
  • Theorem21 - Wednesday, March 13, 2019 - link

    Seriously, stop using RAID. Use ZFS. Rebuild times, recovery + management are all vastly different and far better. Reply
  • PixyMisa - Wednesday, March 13, 2019 - link

    I started using ZFS last year. It's magical. Reply
  • abufrejoval - Thursday, March 14, 2019 - link

    ZFS is a redundant array of inexpensive disks, too, just smarter software to operate it.

    And I run oVirt/GlusterFS for RAIS (redundant array of inexpensive servers).
    Reply
  • piroroadkill - Monday, March 18, 2019 - link

    Your comparison literally makes no sense. One is a name given to a bunch of different redundancy schemes, one is a file system. They can't be compared like that. If you're comparing RAIDZ in ZFS to other types of RAID, then what you're saying starts to make sense, but you're still using a type of RAID. Reply
  • 32TB - Thursday, March 14, 2019 - link

    What are you talking about? I want over 16 TB per drive. F this fake news author! Reply

Log in

Don't have an account? Sign up now