Introduction: Enterprise Storage 101

Since the introduction of x86 based servers at the end of the 20th century, the cost of server hardware has declined rapidly while the performance per watt and performance per dollar has increased rapidly. This pushed the server market to evolve from closed, proprietary, and most importantly extremely expensive mainframe and proprietary RISC servers into today's highly competitive x86 server market. However, the professional storage market is still ruled by the proprietary, legacy systems.

Today, you can get a very powerful server that can cope with most server workloads for something like $5000. Even better, you can run tens of workloads in parallel by virtualizing them. But go to the storage market with four or even five times the budget and you will likely return with a low end SAN.

Worse yet is that there is good chance this expensive device will choke regularly due to the use of a storage intensive application. We quote a market survey conducted in march 2013:

Forty-four percent of respondents said disproportionate storage-related costs were an obstacle preventing them from virtualizing more of their workloads. Forty-two percent said the same about performance degradation or the inability to meet performance expectations.

Note that the study does not mention the percentage of customers stuck in denial :-). The performance per dollar of the average SAN array is mediocre at best, and the storage capacity per dollar is simply awful.

You might think that the hardware inside a SAN is vastly superior to what can be found in your average server, but that is not the case. EMC (the market leader) and others have disclosed more than once that “the goal has always to been to use as much standard, commercial, off-the-shelf hardware as we can”. So your SAN array is probably nothing more than a typical Xeon server built by Quanta with a shiny bezel. A decent professional 1TB drive costs a few hundred dollars. Place that same drive inside a SAN appliance and suddenly the price per terabyte is multiplied by at least three, sometimes even 10! When it comes to pricing and vendor lock-in you can say that storage systems are still stuck in the “mainframe era” despite the use of cheap off-the-shelf hardware.

So why do EMC, NetApp, and the other giants in the storage market charge so much for what is essentially a Xeon based server, an admittedly well designed and reliable storage backplane, and some unreliable and slow performing hard drives? The reason is not some complex market situation that can only be explained by financial experts. No, the key is simply the basic component of a storage system: the unreliable and slow magnetic disk.

As we all know, the magnetic disk is the component that fails most in the data center, and it is by far the slowest core component in modern computers. Building a reliable and somewhat performant storage system based upon such a mediocre storage component can only be done with complex software. And complex software is yet another prime reason why IT services fail. So only a few companies that were able to build a solid reputation gained enough trust to succeed in the storage market.

This is why EMC, NetApp, IBM, and HP rule the storage market today even though they are charging an arm and a leg for a few terabytes of capacity. Professional buyers trust the devices these vendors make and are willing to pay a huge premium just to be sure that they get good reliability and decent but hardly compelling performance and capacity.

The Winds of Change
Comments Locked

60 Comments

View All Comments

  • prime2515103 - Wednesday, August 7, 2013 - link

    I know, it just seems unprofessional. It's a tech article, not a chat room.
  • FunBunny2 - Tuesday, August 6, 2013 - link

    What is always missing from such essays (and this one reads more like a 'Seeking Alpha' pump piece) is a discussion of datastore structure. If you want speed and fewer bytes and your data isn't just videos, then an industrial strength RDBMS with a Organic Normal Form™ schema gets rid of the duplicate bytes. Too bad.
  • DukeN - Tuesday, August 6, 2013 - link

    But has there actually been any disruptions to the top dogs?

    EMC, NetApp, storage from HP/Dell/IBM, Hitachi all have had significant earnings increases yet again.

    So maybe a couple of new startups as well as FusionIO are making money now, but some of the big guys can probably just buy them out and shelf them.
  • davegraham - Tuesday, August 6, 2013 - link

    Look at EMC's acquisition of XtremeI/O...that's a viable competitor that EMC has already been able to integrate as a mainstream product. Oh, and they're also using Virident PCIe cards for server-side flash. ;)
  • DukeN - Wednesday, August 7, 2013 - link

    But is that really disruptive, or business as usual? These guys usually buy up smaller technologies as needed and integrate them if needed. Most of their core business (spinning disks) has remained the same.
  • bitpushr - Friday, August 9, 2013 - link

    XtremIO is still not a shipping product. It is not generally-available. So, I do not think this qualifies as "integrate as a mainstream product".

    Likewise their server-side Flash sales (Project Lightning) have been extremely slow.
  • phoenix_rizzen - Tuesday, August 6, 2013 - link

    If you ditch Windows on the desktop, you can do a lot more for a lot less.

    $22,000 for a Nutanix node to support a handful of virtual desktops? And you still need the VDI client systems on top of that? Pffft, for $3000 CDN we can support 200-odd diskless Linux workstations (diskless meaning they boot off the network, mount all their filesystems via NFS, and run all programs on the local system using local CPU, local RAM, local GPU, local sound, etc). The individual desktops are all under $200 (AMD Athlon-II X3 and X4, 2 GB of RAM, onboard everything; CPU fan is the only moving part) and treated like appliances (when one has issues, just swap it out for a spare).

    No licensing fees for the OS, no licensing fees for 90+% of the software in use, no exorbitant markup on the hardware. And all staff and students are happy with the system. We've been running this setup in the local school district for just shy of 10 years now. Beats any thin-client/VDI setup, that's for sure.
  • turb0chrg - Tuesday, August 6, 2013 - link

    Another vendor doing hybrid storage is Nimble Storage (http://www.nimblestorage.com/). I've looked at their solution and it is quite impressive. It's not cheap though.

    They also claim to be the fastest growing storage vendor!
  • dilidolo - Tuesday, August 6, 2013 - link

    I have 2 of them for VDI, they work fine, but I wouldn't call it enterprise storage.
  • equals42 - Saturday, August 17, 2013 - link

    It's only iSCSI so you better like that protocol.

Log in

Don't have an account? Sign up now