Single Client Performance - CIFS and NFS on Linux

A CentOS 6.2 virtual machine was used to evaluate NFS and CIFS performance of the NAS when accessed from a Linux client. In order to standardize the testing across multiple NAS units, the following parameters were used to mount the NFS and Samba shares:

mount -t nfs NAS_IP:/PATH_TO_NFS_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER

mount -t cifs //NAS_IP/PATH_TO_SMB_SHARE /PATH_TO_LOCAL_MOUNT_FOLDER

Note that these are slightly different from what we used to run in our previous NAS reviews. We have also shifted from IOMeter to IOZone for evaluating performance under Linux. The following IOZone command was used to benchmark the shares:

IOZone -aczR -g 2097152 -U /PATH_TO_LOCAL_CIFS_MOUNT -f /PATH_TO_LOCAL_CIFS_MOUNT/testfile -b <NAS_NAME>_CIFS_EXCEL_BIN.xls > <NAS_NAME>_CIFS_CSV.csv

IOZone provides benchmark numbers for a multitude of access scenarios with varying file sizes and record lengths. Some of these are very susceptible to caching effects on the client side. This is evident in some of the graphs in the gallery below.

Readers interested in the hard numbers can refer to the CSV program output here. These numbers will gain relevance as we benchmark more NAS units with similar configuration.

The NFS share was also benchmarked in a similar manner with the following command:

IOZone -aczR -g 2097152 -U /nfs_test_mount/ -f /nfs_test_mount/testfile -b <NAS_NAME>_NFS_EXCEL_BIN.xls > <NAS_NAME>_NFS_CSV.csv

Some scenarios exhibit client caching effects, and these are evident in the gallery below.

The IOZone CSV output can be found here for those interested in the exact numbers.

A summary of the bandwidth numbers for various tests averaged across all file and record sizes is provided in the table below. As noted previously, some of these numbers are skewed by caching effects. A reference to the actual CSV outputs linked above make the entries affected by this effect obvious.

Netgear ReadyNAS 716 - Linux Client Performance (MBps)
IOZone Test CIFS NFS
Init Write 76 16
Re-Write 76 16
Read 32 120
Re-Read 32 121
Random Read 19 51
Random Write 73 19
Backward Read 19 42
Record Re-Write 743 401
Stride Read 28 82
File Write 76 16
File Re-Write 75 17
File Read 23 84
File Re-Read 22 87

 

Single Client Performance - CIFS and iSCSI on Windows Multi-Client Performance - CIFS
Comments Locked

24 Comments

View All Comments

  • Guspaz - Wednesday, January 1, 2014 - link

    Yikes, that's a highly questionable decision, to go with btrfs instead of ZFS as the default file system. ZFS has been in production use for seven years now, proven through widespread deployments and available on every *nix platform you can think of, while btrfs is still beta quality (without even an official stable release) and nowhere near feature-competitive with ZFS...
  • JDG1980 - Wednesday, January 1, 2014 - link

    Agreed. This is a full-fledged Xeon PC with ECC RAM, so why not go with ZFS? It would seem to be the obvious choice for a high-quality, time-tested software RAID system.

    By the way, it would really be better if you listed the suggested retail price on the first page of reviews along with the other specs. (A quick Google search seems to indicate that the street price is $2500-$3000.)
  • Runiteshark - Wednesday, January 1, 2014 - link

    Probably because it takes a bit more effort to get ZFS running in Linux than btrfs, but not that much. It recently went stable and is working just fine on a 72 bay Supermicro chassis I have in test for the past 3 months. All this being said, why didn't they just go with a BSD solution?
  • nafhan - Thursday, January 2, 2014 - link

    While, BTRFS has been supported as a root file system in SLE and Oracle Linux since 2012. ZFS: not available from the vendor on either (even though Solaris is owned by Oracle). That's probably it right there.
  • shodanshok - Friday, January 3, 2014 - link

    I agree. While BTRFS is quite stable now, considering the critical role assigned to a filesystem I would go with a FS with a proven track record (and fsck). Moreover being a CoW filesystem, BTRFS tend to be extremely fragmentation prone in some circumstances, basically everytime a file rewrite is required, for example a database or a virtual machine (but I think that a similar NAS units is primarily assigned with archiving role).
  • SirGCal - Wednesday, January 1, 2014 - link

    Yup, I have two 8-disk systems myself. One running hardware LSI controller for RAID 6 and one using ZFS for the same effective protection. Sure the hardware controller is actually a tiny bit faster at hard reads, but for the $600 price tag, so what. All of my current systems are going to be ZFS. These arrays in a box are interesting until they decide to go with some other pooling system... If there is a real comparable reason and argument for BTRFS instead of ZFS, I'd like to see it.
  • Runiteshark - Wednesday, January 1, 2014 - link

    I tested btrfs recently with a large disk array (read 45 4TB drives) and the performance was very poor. Ended up going with JFS and shunned XFS because it's not stable in the event of power issues.
  • shodanshok - Friday, January 3, 2014 - link

    Hi,
    from my understanding JFS and JFS2 are more or less unsupported from some time now.

    What problem did you have with XFS? It is designed to manage the exact case you describe: a lot of space spread over a lot of spinning disks. When using XFS, the only two thing that can lead to data loss are:
    1. no barrier/FUA support in the disk/controller combo
    2. an application that rewrite files with truncate and do _not_ use fsyncs

    Case n.1 is common to all filesystem: if your disk lies about cache flush, then no filesystem can save you. The only thing that can somewhat lessen the risk is journal checksumming, that is implemented in both XFS, EXT4 and BRTFS, but I don't know about JFS.

    Case n.2 is really an application shortcoming, but EXT4 and BTRFS choice here are the more sensible one: detect such corner case and apply a work-around. Anyway, with application the properly use fsync, XFS is rock stable.

    Regards.
  • Runiteshark - Wednesday, January 1, 2014 - link

    On one hand, I'm happy that 10g is becoming more prevalent slowly for the con/prosumer grade market, however products like this make my head hurt. The performance that you were able to get out of this host were nothing short of embarrassing, and could of easily been handled by a single gigabit link. I think this primarly stems from vendors still using software RAID without using good quality HBAs. You can most certainly have a fantastic software solution that is high performance without a real RAID controller or even a high end HBA, however it requires you use ceph, or ZFS.

    The performance you are seeing out of this is actually very similar to a HP Microserver that I have running on FreeNAS with 2GB of ram, LAGG gigabit ports, 4x4TB 7200rpm Seagates + 32GB USB3 OS drive, granted the entire unit cost no more than $1800, and only has 4 slots instead of 6. Without a doubt if I was going to build something bigger, I'd use a Supermicro X9DR7-TF+, same as what I use in production for $800, get a decent chassis, the LSI BBU and have support for up to 16 drives with 2 10G ports with an Intel X540 chipset, which all toghether would still be significantly less than this solution, and obviously blow the performance of this out of the water.
  • hpglow - Wednesday, January 1, 2014 - link

    Runiteshark not good at reading or convertng bits to bytes? With some of the tests pushing over 600 MB/sec a 1G ethernet port would be saturated more than 4 times over not including packet overhead. A 1Gb ethernet port is good for only 125 MB/sec.

Log in

Don't have an account? Sign up now