CloudFounders: No More RAID

CloudFounders, a startup of former Terremark and SUN execs, also leverages a flash cache, but the building blocks are very different. Just like Nutanix, there is a VSA (Virtual Storage Appliance) that tries to make the best out of a local flash cache. The cool thing is that the backend, the second tier of storage, is not a traditional RAID based volume. The backend is either an object storage initiator that links to Amazon S3 or a storage device based upon erasure encoding called Distribute Storage System (DSS). Let's start with the DSS backend.

DSS is an object oriented storage system that uses “Bitspread”, an advanced and flexible erasure encoding system developed by the people of Amplidata. Amplidata is a startup with a mix of Belgian and US based infrastructure experts. Some of the directors are working for both CloudFounders and Amplidata. But there is a solid technical reason why CloudFounders chose to go with the Amplidata storage system. Bitspread is meant to be the “big storage alternative” to RAID.

As you probably know or have experienced hands-on, the current RAID implementations—RAID 5 and RAID 6—have reached their limitations now that we have terabyte disks. A few terabyte disks in RAID 5 can take days to rebuild. The result is that the RAID array performance and reliability is heavily degraded. RAID 6 is more reliable (although hardly 100%) but is not exactly a good performer for writes, which is another reason why VDI does not work well on a low end or midrange SAN.

“Bitspread” erasure encoding, also called Forward Error Correction Code (FEC), encodes data in “check-blocks”. The beauty is that you can configure the durability policy. In other words you can choose over how many disks these check-blocks should be spread and how many check-blocks you can lose before it becomes a problem. For example you could ask it to spread the datablock over 18 drives and tell the codec to make sure you can recover the original datablock from 12 check-blocks. So it's only if you lose more than 6 drives at once that you lose your data. As the codec requires only 12 of the check blocks to rebuild the original data object, a failure of two drives does not mean the rebuild should happen urgently. The rebuild can be done in the background at a very slow pace while the reliability stays high. You can also have the check-blocks spread over several storage modules, ensuring that you even survive a failure of a complete disk enclosure.

Bitspread: original data (yellow) is split up, encoded with high redundancy (green) and then spread over many disks and enclosures.

For those who are not convinced that the small startup Amplidata is onto something: Intel and Dr. Sam Siewert of Trellis Logic explain in this paper why it can even be mathematically proven that the Reed-Solomon based erasure codes of RAID 6 are a dead end road for large storage systems. The paper concludes:

"Amplida's Bitspread is an efficient, scalable and practical alternative to the stop-gap of combined RAID levels like 6+1."

And that is exactly the reason why CloudFounders chose to build their storage system on the Amplidata backend.

The DSS based on “Bitspread” works with objects and is thus not a block device. A volume driver must be installed that converts the DSS into a block device. This way the hypervisor can connect to an iSCSI target that is running on top of the volume driver, as an iSCSI target requires a block device and does not recognize the format of the DSS.

Bitspread is a lot more CPU intensive and needs more storage room than traditional RAID algorithms. To reduce the CPU impact, Amplidata leverages the SSE 4.2 capabilities of the latest Xeons. As Bitspread copes so well with disk failures, it is natural to use relatively slow SATA disks, which negates the capacity disadvantage compared to RAID 6. Decent media transfer can still be achieved as the DSS typically spreads the check-blocks over many disks.

Nutanix: No More SAN CloundFounders: Cloud Storage Router
Comments Locked


View All Comments

  • prime2515103 - Wednesday, August 7, 2013 - link

    I know, it just seems unprofessional. It's a tech article, not a chat room.
  • FunBunny2 - Tuesday, August 6, 2013 - link

    What is always missing from such essays (and this one reads more like a 'Seeking Alpha' pump piece) is a discussion of datastore structure. If you want speed and fewer bytes and your data isn't just videos, then an industrial strength RDBMS with a Organic Normal Form™ schema gets rid of the duplicate bytes. Too bad.
  • DukeN - Tuesday, August 6, 2013 - link

    But has there actually been any disruptions to the top dogs?

    EMC, NetApp, storage from HP/Dell/IBM, Hitachi all have had significant earnings increases yet again.

    So maybe a couple of new startups as well as FusionIO are making money now, but some of the big guys can probably just buy them out and shelf them.
  • davegraham - Tuesday, August 6, 2013 - link

    Look at EMC's acquisition of XtremeI/O...that's a viable competitor that EMC has already been able to integrate as a mainstream product. Oh, and they're also using Virident PCIe cards for server-side flash. ;)
  • DukeN - Wednesday, August 7, 2013 - link

    But is that really disruptive, or business as usual? These guys usually buy up smaller technologies as needed and integrate them if needed. Most of their core business (spinning disks) has remained the same.
  • bitpushr - Friday, August 9, 2013 - link

    XtremIO is still not a shipping product. It is not generally-available. So, I do not think this qualifies as "integrate as a mainstream product".

    Likewise their server-side Flash sales (Project Lightning) have been extremely slow.
  • phoenix_rizzen - Tuesday, August 6, 2013 - link

    If you ditch Windows on the desktop, you can do a lot more for a lot less.

    $22,000 for a Nutanix node to support a handful of virtual desktops? And you still need the VDI client systems on top of that? Pffft, for $3000 CDN we can support 200-odd diskless Linux workstations (diskless meaning they boot off the network, mount all their filesystems via NFS, and run all programs on the local system using local CPU, local RAM, local GPU, local sound, etc). The individual desktops are all under $200 (AMD Athlon-II X3 and X4, 2 GB of RAM, onboard everything; CPU fan is the only moving part) and treated like appliances (when one has issues, just swap it out for a spare).

    No licensing fees for the OS, no licensing fees for 90+% of the software in use, no exorbitant markup on the hardware. And all staff and students are happy with the system. We've been running this setup in the local school district for just shy of 10 years now. Beats any thin-client/VDI setup, that's for sure.
  • turb0chrg - Tuesday, August 6, 2013 - link

    Another vendor doing hybrid storage is Nimble Storage ( I've looked at their solution and it is quite impressive. It's not cheap though.

    They also claim to be the fastest growing storage vendor!
  • dilidolo - Tuesday, August 6, 2013 - link

    I have 2 of them for VDI, they work fine, but I wouldn't call it enterprise storage.
  • equals42 - Saturday, August 17, 2013 - link

    It's only iSCSI so you better like that protocol.

Log in

Don't have an account? Sign up now