ZFS - Building, Testing, and Benchmarking
by Matt Breitbach on October 5, 2010 4:33 PM EST- Posted in
- IT Computing
- Linux
- NAS
- Nexenta
- ZFS
If you are in the IT field, you have no doubt heard a lot of great things about ZFS, the file system originally introduced by Sun in 2004. The ZFS file system has features that make it an exciting part of a solid SAN solution. For example, ZFS can use SSD drives to cache blocks of data. That ZFS feature is called the L2ARC. A ZFS file system is built on top of a storage pool made up of multiple devices. A ZFS file system can be shared through iSCSI, NFS, and CFS/SAMBA.
We need a lot of reliable storage to host low cost websites at No Support Linux Hosting. In the past, we have used Promise iSCSI solutions for SAN based storage. The Promise SAN solutions are reliable, but they tend to run out of disk IO long before they run out of disk space. As a result, we have been intentionally under-utilizing our current SAN boxes. We decided to investigate other storage options this year in an effort to improve the performance of our storage without letting costs get completely out of hand.
We decided to spend some time really getting to know OpenSolaris and ZFS. Our theory was that we could build a custom ZFS based server for roughly the same price as the Promise M610i SAN, and the ZFS based SAN could outperform the M610i at that price point. If our theory proved right, we would use the ZFS boxes in future deployments. We also tested the most popular OpenSolaris based storage solution, Nexenta, on the same hardware. We decided to blog about our findings and progress at ZFSBuild.com, so others could benefit from anything we learned throughout the project.
105 Comments
View All Comments
Penti - Wednesday, October 06, 2010 - link
And a viable alternative still isn't available how is Nexenta and the community suppose to get driver support and support for new hardware there, when Oracle has closed the development kernel (SXDE is closed source), meaning that they maybe just maybe can use the retail Solaris 11 kernel if it's released in a functioning form that can be piped in with existing software and distro. They aren't going to develop it themselves and the vendors have no reason giving the code/drivers to anybody but Oracle. Continuing the OpenSolaris kernel means creating a new operating system. It means you won't get the latest ZFS updates and tools any more, at least not till they are in the normal S11 release. Means you can't expect the latest driver updates and so on either. You can continue to use it on todays hardware, but tomorrow it might be useless, you might not find working configurations.It's not clear that Nexenta actually can develop their own operating system, rather then just a distro, it means they have to create their own OS with their own kernel eventually. With their own drivers and so on. And it's not clear how much code Oracle will let slip out. It's just clear that they will keep it under wraps till official releases. It's however clear that there won't be any distro for them to base it on and any and all forks would be totally dependent on what Nexenta (Illuminos) manage to do. It will quickly get outdated without updates flowing all the time, and they came from Sun. Reply
andersenep - Wednesday, October 06, 2010 - link
OpenIndiana/Illumos runs the same latest and greatest pool/zfs versions as the most recent Solaris 10 update.Work continues on porting newer pool/ZFS versions to FreeBSD which has plenty of driver support (better than OpenSolaris ever did).
A stated goal of the Illumos project is to maintain 100% binary compatibility with Solaris. If Oracle decides the break that compatibility, intentionally or not, it will truly become a fork. Development will still continue.
Even if no further development is made on ZFS, it's still an absolutely phenomenal filesystem. How many years now has Apple been using HFS+? FAT is still around in everything. If all development on ZFS stopped today, it would still remain an absolutely viable filesystem for many years to come. There is nothing else currently out there that even comes close to its feature set.
I don't see how ZFS being under Oracle's control makes it any worse than any other open source filesystem. The source is still out there, and people are free to do what they want with it within the CDDL terms.
This idea that just because the OpenSolaris DISTRO has been discontinued, that everything that went into it is no longer viable is silly. It is like calling Linux dead because Mandriva is dead. Reply
Guspaz - Wednesday, October 06, 2010 - link
Thanks for mentioning OpenIndiana. I've been eagerly awaiting IllumOS to be built into an actual distribution to give me an upgrade path for my home OpenSolaris file server, and I look forward to upgrading to the first stable build of OpenIndiana.I'm currently running a dev build of OpenSolaris since the realtek network driver was broken in the latest stable build of OpenSolaris (for my chipset, at least). Reply
geekfool - Wednesday, October 06, 2010 - link
the first step is to build a base template that all of the virtual machines will start from. If you were planning to host a lot of Linux virtual machines, you would build the base template by installing Linux. When you get to the step where you would normally configure the server, you would shut off the virtual machine. At that point, you would have the base template ready. Each additional virtual machine would simply be chained off the base template.what hypervisor permit this? latest esxi? xenserver? hyper-v? Reply
Mattbreitbach - Wednesday, October 06, 2010 - link
I believe all of the current Hypervisors support this. Hyper-V does, as does XenServer. I have not done extensive testing with ESXi, but I would imagine that it supports it also. Replygeekfool - Wednesday, October 06, 2010 - link
and what is this features in this HyperV and Xen? vbox also support this?I know VMware Lab Manager but I do not think this is the way.
seems to capture something like a snapshot, but how do I create new vm on them is entirely foreign to me. Reply
Mattbreitbach - Wednesday, October 06, 2010 - link
In Hyper-V it is called a Differencing disk - you have a parent disk that you build, and do not modify. You then create a "differencing disk". That disk uses the parent disk as it's source, and writes any changes out to the differencing disk. This way you can maintain all core OS files in one image, and write any changes out to child disks. This allows the storage system to cache any core OS components once, and any access to those core components comes directly from the cache.I believe that Xen calls it a differencing disk also, but I do not currently have a Xen Hypervisor running anywhere that I can check quickly. Reply
joeribl - Wednesday, October 06, 2010 - link
"Nexenta is to OpenSolaris what OpenFiler or FreeNAS is to Linux."FreeNAS has always been FreeBSD based, not Linux. It does however provide ZFS support. Reply
Mattbreitbach - Wednesday, October 06, 2010 - link
I should have caught that - thanks for the info. I've edited the article to reflect as such. Replyvermaden - Wednesday, October 06, 2010 - link
... with deduplication and other features, here You can grab an ISO build or a VirtualBox apliance here: http://blog.vx.sk/archives/9-Pomozte-testovat-ZFS-...It would be great to see how FreeBSD performs (8.1 and 9-CURRENT) on that hardware, I can help You configure FreeBSD for these tests if You would like to, for example, by default FreeBSD does not enables AHCI mode for SATA drives which increases random performance a lot.
Anyway, great article about ZFS performance on nice piece of hardware. Reply