ZFS - Building, Testing, and Benchmarking

Strictly from a price vs. performance standpoint, the OpenSolaris ZFS server is a huge success.  The test of 4k random performance with 66% writes and 33% reads is one that really excites us.  We have historically graphed a lot of performance data about our cloud environment using MRTG, and this test most accurately describes our real world access patterns in our web hosting environment.  At a load of 33, OpenSolaris logged nearly triple the performance compared to the Promise M610i.  If we can deploy the OpenSolaris box into production and actually see this level of performance, we will be thrilled. 

We will obviously consider using Nexenta because Nexenta has the LED’s and notifications working.  Nexenta did not deliver as much performance as OpenSolaris did.  In the 4k random 66% write 33% read test, Nexenta Enterprise managed to deliver about 90% of the performance of unmodified OpenSolaris.  In some 32k tests, Nexenta delivered even less performance.  It is tough to justify the cost of the Nexenta Enterprise license when it performs slower than free OpenSolaris.  If you need a support path, then Nexenta Enterprise may be worth it.

When we started this project, our goal was to build a ZFS based storage solution that could match the price of a Promise M610i SAN, yet measurably exceed the performance.  We believe we have succeeded in doing exactly that.  Our ZFS server can be built for about the same price as the Promise M610i.  The performance of the OpenSolaris ZFS server at high loads was anywhere from double to quadruple the performance of the Promise solution in most tests at nearly the same cost.  We deploy additional SAN boxes each year.  Based on the performance of this test, our next SAN boxes will be ZFS based.