In our last SandForce SSD roundup I talked about how undesirable the consumer SSD market is, at least for those companies who don't produce their own controllers and/or NAND. There's a downward trend in NAND and SSD pricing, which unfortunately squeezes drive manufacturers as they compete for marketshare. The shrinking margins in the consumer SSD space will ultimately drive companies out of the business, consolidating power in those companies who are fine operating with slim margins. There are other effects, such as insufficient validation testing that result from this price crunch.

Public companies are under an even greater pressure to maintain high profit margins. Investors don't care about good product, they care about good returns. What is a public SSD manufacturer like OCZ to do? Go after the enterprise market of course.

OCZ has quietly addressed the enterprise SSD space for a while now. Today you can custom order an OCZ Deneva 2 SSD which is an enterprise focused SF-2000 based solution. OCZ's enterprise drives are fully customizable down to the controller, firmware and NAND type on-board. If you want an SF-2000 drive with SAS support and SLC NAND, OCZ will build it for you.

In the enterprise segment where 1U and 2U servers are common, PCI Express SSDs are very attractive. You may not always have a ton of 2.5" drive bays but there's usually at least one high-bandwidth PCIe slot unused. The RevoDrive family of PCIe SSDs were targeted at the high-end desktop or workstation market, but for an enterprise-specific solution OCZ has its Z-Drive line.

We introduced the Z-Drive R4 in our Computex coverage earlier this year - it's a beast. The Z-Drive R4 is a multi-controller PCIe solution that uses either 4 or 8 SF-2000 controllers behind a SAS-to-PCIe 2.0 x8 controller. The breakdown is as follows:

OCZ Z-Drive R4 Lineup
  CM84 CM88 RM84 RM88
Capacities 300/600/1200GB 800/1600/3200GB 300/600/1200GB 800/1600/3200GB
NAND MLC MLC MLC MLC
Interface PCIe Gen 2 x8 PCIe Gen 2 x8 PCIe Gen 2 x8 PCIe Gen 2 x8
Form Factor Half Height PCIe Full Height PCIe Half Height PCIe Full Height PCIe
Dimensions LxWxH 168.55 x 68.91 x 17.14 mm 242 x 98.4 x 17.14mm 168.55 x 68.91 x 17.14 mm 242 x 98.4 x 17.14mm
SSD Controllers 4 x SF-2282 8 x SF-2282 4 x SF-2582 8 x SF-2582
Power Failure Protection N N Y Y
Max Read 2000MB/s 2800MB/s 2000MB/s 2800MB/s
Max Write 2000MB/s 2800MB/s 2000MB/s 2800MB/s
Max Random Read 250K IOPS 410K IOPS 250K IOPS 410K IOPS
Max Random Write 160K IOPS 275K IOPS 160K IOPS 275K IOPS

The xM84s are half height solutions with four controllers while the xM88s are full height with eight controllers. The C-series use SF-2282 controllers while the R-series use SF-2582. The main difference there is the support for power failure protection. The R-series boards have an array of capacitors that can store enough charge to flush all pending writes to the NAND arrays in the event of a power failure. The C-series boards do not have this feature.

Despite the spec table above, OCZ also offers customized solutions as I mentioned above. The table above simply highlights the standard configurations OCZ builds.

For today's review OCZ sent us a 1.6TB Z-Drive R4 CM88. We have a preproduction board that has a number of stability & compatibility issues. OCZ tells us these problems will be addressed in the final version of the drives due to ship in the coming weeks. OCZ expects pricing on this board to be somewhere in the $6 - $7/GB range depending on configuration. Doing the math that works out to be anywhere between $9600 - $11200 for this single SSD. OCZ typically sells SF-2281 based SSDs at around $2/GB, even accounting for the extra controllers on-board there should be a hefty amount of profit for OCZ in the selling price of these Z-drives.

As with the RevoDrive X2 models the Z-Drive R4 CM88 uses two PCBs to accommodate all of its controllers. Each PCB is home to four SF-2282 controllers and 64 Intel 25nm MLC NAND devices (8 controllers, 128 devices total). Each NAND device has two 8GB die inside, which works out to be 2048GB of NAND on-board. This is an absolutely insane amount of NAND for a single drive. Remember each 8GB MLC NAND die (25nm) is 167mm2, which means this Z-Drive R4 has 42752mm2 of 25nm silicon on-board. A single 300mm wafer only has a surface area of 70685mm2 (even less is usable), which means it takes more than half of a 300mm 25nm MLC NAND wafer to supply the flash for just one of these drives. Roughly 27% of the NAND is set aside as spare area, exposing 1490GiB to the OS.

Thanks to the eight SF-2282 controllers and tons of NAND in close proximity, OCZ requires 100 CFM of air to properly cool the Z-Drive R4. This is clearly a solution for a rack mounted server.


It's OCZ branded but this is a Marvell SAS controller - the same driver works on the RevoDrive 3 X2 and the Z-Drive R4

OCZ continues to use its VCA 2.0 architecture on the Z-Drive R4. Details are still vague but OCZ claims to have written its own driver and firmware for the Marvell SAS controller on the Z-Drive R4 that allows it to manage redirection of IOs based on current controller queue depths rather than a dumb RAID-stripe. The driver accumulates IOs and redestributes them to the drive's controller array, to some degree, dynamically. OCZ's VCA also allows TRIM to be passed to the array although Microsoft's OSes won't pass TRIM to SCSI/SAS drives. You can use OCZ's Toolbox to secure erase the drive but there's no real-time TRIM available, this is a Microsoft limitation that impacts all SAS based drives.

The Test

This is going to be a bit of a disappointing set of performance graphs as it is our first real foray into the world of enterprise SSD testing. You will see results from the RevoDrive 3 X2 as well as a single Vertex 3 and Intel's X25-E, however we have no other high-end PCIe SSDs at our disposal for comparison. We have put in a request to FusionIO for a competing solution however it appears to have fallen on deaf ears. We will use this review to begin assembling our enterprise SSD data and hopefully in the coming weeks and months we'll be able to have a set equivalent to what we have in the consumer space.

We also had to run the Z-Drive R4 on our old X58 testbed instead of our H67 system as it wouldn't complete any tests on our H67 platform. OCZ attributes this to an issue with the preproduction Z-Drive boards which it says will be corrected by the time mass production begins.

CPU

Intel Core i7 965 running at 3.2GHz (Turbo & EIST Disabled) for Enterprise Bench

Intel Core i7 2600K running at 3.4GHz (Turbo & EIST Disabled) - for AT SB 2011, AS SSD & ATTO

Motherboard:

Intel DX58SO (Intel X58)

Intel H67 Motherboard

Chipset:

Intel X58

Intel H67
Chipset Drivers:

Intel 9.1.1.1015 + Intel RST 10.2

Memory: Qimonda DDR3-1333 4 x 1GB (7-7-7-20)
Video Card: eVGA GeForce GTX 285
Video Drivers: NVIDIA ForceWare 190.38 64-bit
Desktop Resolution: 1920 x 1200
OS: Windows 7 x64

 

Random Read/Write Speed
POST A COMMENT

57 Comments

View All Comments

  • lorribot - Wednesday, September 28, 2011 - link

    You cant raid it in software but you could raid several of them if you have deep pockets.
    The point is why buy a 1.6 or 3.2 TB SSD when you can buy 10 x 320 gb SSDs and (possibly) get better performance for less cost?
    Reply
  • bjacobson - Tuesday, September 27, 2011 - link

    I think I've mentioned this before but can you load up a Windows 7 installation with 30 or so startup programs and compare the startup time difference between this and a harddrive?
    A video of this would be be even impressive.
    Reply
  • ckryan - Tuesday, September 27, 2011 - link

    I've been going through some issues with a 2281 drive with Toggle nand. I'm basically writing 11TB a day to it and under these conditions I can't get too many hours in between crashes. I'm of the opinion that the latest FW has helped most out, but clearly my experience shows that the 2281, when perfected, will be unstoppable in certain workloads, but for now all SF users are going to have some problems. If the problems are predictable you can compensate, but if they're random, well SF controllers aren't the only things that have problem with randomness. I knew it was a possibility, and that normal users won't abuse their drives as much, but I have to wonder if OCZ can make an enterprise drive problem free, why can't they make consumer SF drives better? The SF problem is the OCZ problem... OWC doesn't have the same perception issues, but is using the same hardware (Mushkin,Patriot, etc). As much as I like OCZ, they've done some questionable things in the past, and not just swapping cheap flash in SF1200 drives. Hopefully they can overcome the problems they're having with Sandforce and their Arrowana stuff, release a problem free next gen Indilinx controller, and then call it a day. Oh yeah, quit using those stupid plastic chassis. Reply
  • jalexoid - Tuesday, September 27, 2011 - link

    Considering these devices are more likely to find themselves in a machine running something else than a desktop system, why not test them on another OS? Reply
  • sanguy - Wednesday, September 28, 2011 - link

    OCZ's standard line "It's only affecting 0.01% of the deployed units and we're working on a fix....." doesn't work in the enterprise market. Reply
  • josephjpeters - Wednesday, September 28, 2011 - link

    These are PCIe. Most of the "issues" come from SATA drives because mobo makers are having issues with their SATA ports. Reply
  • p05esto - Wednesday, September 28, 2011 - link

    I'll admit, I'm now too lazy to even read....it's getting bad. I just want to push the "play" button while I sit back eating Cheetos and rubbing my tummy. Get into my tummy little Cheeto, get into my brain little ssd review,... same line of thinking really, whatever is easiest.

    Great review though, seriously.
    Reply
  • alpha754293 - Wednesday, September 28, 2011 - link

    If you want to really test it'and validate it's long term reliability, you pretty much need to do what enterprise customers do. Run the SSD, but always keep a backup of it somewhere, like you said.

    That being said though, if you've got TWO backup copies of it, you can actually run a parity check on it (pseudo-checksum) and determine its error rate.

    Also, you didn't run HDTach on it. Given that it's tied together with a Marvell SAS controller, NOT being able to run TRIM on it, I would presume, will give it performance issues in the long run.

    To do the error checking, you'll probably have to put this thing in a Solaris system running ZFS so you can mimic the CERN test. And if you actually read/write continuously to it, at the same level in terms of the sheer volume of data, other SSD/NAND-specific issues might start to pop up like wear levelling, etc. I would probably just run the read/write cycle for an entire month, where it periodically deletes some data, rewrite new data, etc. At the end of the month, make the two mirror backups of it. And then run it again. Hopefully you'd be able to end up at some identical endpoint after PBs of read/write ops that you can run both the block level and volume level checksum on.

    But as a swap drive, this would be BLAZINGLY fast.
    Reply
  • perrydoell - Wednesday, September 28, 2011 - link

    You say "We have a preproduction board that has a number of stability & compatibility issues."

    This is the enterprise space. Things MUST WORK RELIABLY. How can you even review unstable products? I expect better from Anandtech.

    I cannot take OCZ seriously either. An unstable product is NOT for the enterprise. Also, check the negative reviews at NewEgg. Ouch.
    Reply
  • josephjpeters - Wednesday, September 28, 2011 - link

    Where is the R4 listed on Newegg? Reply

Log in

Don't have an account? Sign up now