File Server Builder's Guide
by Zach Throckmorton on September 4, 2011 3:30 PM ESTHard drives
One of the most frequently asked questions I hear is 'what's the most reliable hard drive?' The answer to this question is straightforward - the one that's backed up frequently. Home file servers can be backed up with a variety of devices, from external hard drives to cloud storage. As a general guideline, RAID enhances performance but it is not a backup solution. Some RAID configurations (such as RAID 1) provide increased reliability, but others (such as RAID 0) actually decrease reliability. A detailed discussion of different kinds of disk arrays is not within the scope of this guide, but the Wikipedia page is a good place to start your research if you're unfamiliar with the technology.
As for hard drive reliability, every hard drive can fail. While some models are more likely to fail than others, there are no authoritative studies that implement controlled conditions and have large sample sizes. Most builders have preferences - but anecdotes do not add up to data. There are many variables that all affect a drive's long-term reliability: shipping conditions, PSU quality, temperature patterns, and of course, specific make and model quality. Unfortunately, as consumers we have little control over shipping and handling conditions until we get a drive in our own hands. We also generally don't have much insight into a specific hard drive model's quality, or even a manufacturer's general quality. However, we can control PSU quality and temperature patterns, and we can use S.M.A.R.T. monitoring tools
One of the most useful studies on hard drive reliability was presented by Pinheiro, Weber, and Barroso at the 2007 USENIX Conference on File and Storage Technologies. Their paper, Failure trends in a large disk drive population, relied on data gleaned from Google. So while the controls are not perfect, the sample size is enormous, and it's about as informative as any research on disk reliability. The PDF is widely available on the web and is definitely worth a read if you've not already seen it and you have the time (it's short at only 12 pages with many graphs and figures). In sum, they found that SMART errors are generally indicative of impending failure - especially scan errors, reallocation counts, offline reallocation counts, and probational counts. The take home message: if one of your drives reports a SMART error, you should probably replace it and send it in for replacement if it's under warranty. If one of your drives reports multiple SMART errors, you should almost certainly replace it as soon as possible.
From Pinheiro, Weber, and Barroso 2007. Of all failed HDDs, more than 60% had reported a SMART error.
Pinheiro, Weber, and Barroso also showed how temperature affects failure rates. They found that drives operating at low temperatures (i.e. less than 75F/24C) actually have the highest (by far) failure rates, even greater than drives operating at 125F/52C. This is likely an irrelevant point to many readers, but for those of us who live further up north and like to keep our homes at less than 70F/21C in the winter, it's an important recognition that colder is not always better for computer hardware. Of use to everyone, the study showed that the pinnacle of reliability occurs around 104F/40C, from about 95F/35C to 113F/45C.
From Pinheiro, Weber, and Barroso 2007. AFR: Annualized Failure Rate - higher is worse!
Given the range of temperatures that hard drives appear to function most reliably at, it might take some experimentation in any given case to get a home file server's hard drives in an ideal layout.
So rather than answering what specific hard drive models are the most reliable, we recommend you do everything you can to prevent catastrophic failure by using quality PSUs, maintaining optimal temperatures, and paying attention to SMART utilities. For such small sample sizes as a home file server necessitates, the most important factor in long-term HDD reliability is probably luck.
Pragmatically, low-rpm 'green' drives are the most cost-effective storage drives. Note that many of the low-rpm drives are not designed to operate in a RAID configuration - be sure to research specific models. The largest drives currently available are 3TB, which can now be found for as little as $110. The second-largest capacity drives at 2TB generally offer the best $/GB ratio, and can regularly be found for $70 (and less when on sale or after rebate). 1TB drives are fine if you don't need much space, and can sometimes be found for as little as $40.
Facebook
Twitter
RSS
152 Comments
View All Comments
masterbm - Thursday, September 8, 2011 - link
If was to build to today I would think about uses i3 or low-end i5 sandy-bridge. The cpu was pick because at the time I thought it was the best bang for the bang for what I need it for. The machine has not let my down yet. My back file server ie older media center box houses the old drives from increase the need for more space. The 750 has stayed because it has the size needed to handle is responsibility/. All drive have certain things that hold and when it get filled it is upgraded.MartenKL - Monday, September 12, 2011 - link
I would like my WHS2011 do realtime transcode of 1080p streams with 5.1 channel sound to my xboxes and ps3. Would the 2500T suffice?noxplague - Sunday, October 2, 2011 - link
First, I think most of these comments became focused on SMB/Midmarket type concerns. This guide was clearly aimed on the pro-sumer looking to solve his data proliferation issues. Not everyone needs enterprise style RAID.With the help of this guide I build the following WHS 2011:
Fractal Design Array R2 Black Aluminum Mini-ITX Desktop Computer Case 300W SFX PSU
Foxconn H67S LGA 1155 Intel H67 HDMI SATA 6Gb/s Mini ITX
Intel Pentium G620 Sandy Bridge 2.6GHz
2 X HITACHI Deskstar 7K3000 2TB 7200 RPM 64MB Cache SATA 6.0Gb/s 3.5" (RAID1)
Kingston HyperX 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3
Here are some of the scenarios I have enabled with this build:
Adobe Lightroom 3 on a Mac and on 2 PCs - this is really handy because I can store all photos on the server and have the library on each computer link to the 1 version of the truth. The only limitation is I cannot edit photos on the road that are back on the server.
iTunes - I have an Apple TV and iPad, carefully configuring iTunes on the server and on my clients I am able to download to have files on the server and still sync/stream everything to my different devices.
Zune & WP7 - ditto the above. I need to solve the album art issues across all of these devices however.
Mac - They have no problem accessing the files and RDC works just as well as on Windows.
Current problem: The only scenario that isn't working well is playing any WMV from the server. The "streaming" is painfully slow. This does not make sense to me because Quicktime videos on the server play on the Macs without a problem over wifi. Movie streaming to the Apple TV is seamless as well.
My server is connected to my Apple Airport extreme (4th generation) via Gigabit ethernet.
I found the guide helpful in finding good parts, in particular the case. I wish more time had been spent on scenarios and use cases with the server. Also the HDD section was useless. What would be much more practical would be advise on a decent setup. I decided to use the Intel RAID 1 with identical drives, but I have lots of questions about how to best maintain these and also the best way to consider adding new drives when needed. I'm trying to figure out an automated way to monitor the SMART status and have my server e-mail me if any drive has an issue. An brief overview of different backup methodologies would've been useful as well. Another useful discussion would be on UPS systems and how to configure your server to power down in the case of power loss.
Thanks for the guide and hope to see more on this topic in the future!
pacomcfriendly - Saturday, October 15, 2011 - link
This has been the best OS experience I've found for my own home fileserver. Its built on opensolaris, super easy to work with, free, and zfs / zpool is fantastic.wiz329 - Saturday, December 10, 2011 - link
This is a very beginner question, but what is the best way to access such a file server over the internet?I am looking to either use NAS or build such a home file server to store media. Over LAN, it seems pretty straightforward, just connect to your router. How would you go about accessing/streaming over WAN?
marcus77 - Saturday, October 6, 2012 - link
euroNAS would be also worth looking. They are offering storage software that is more for business use but they also offer technical assistance which is nessesary if something goes wrong. Also they have some advanced features such as storage cluster. http://www.euronas.comAmar7 - Monday, December 3, 2012 - link
Mr. Throckmorton,Great File Server Build Guide... Any update to hardware suggestions? Some of the parts are no longer available. Would love some ideas on a budget home file server.
Much thanks.
buxe2quec - Friday, December 7, 2012 - link
I would put in the list of possible operating systems also NexentaStor, it'a a very polished OpenSolaris/Illumos based NAS distribution free for personal use.war59312 - Monday, December 10, 2012 - link
Would love a 2012 update for this guide?StoatWarbler - Tuesday, October 22, 2013 - link
I'd love to see a 2013 update, especially for tower cases which can take a bunch of hotswap drive racks (My current ZFS array has 20 2Tb drives + 2 SSDs + OS drives. Yes, I'm barking mad and I enjoy it.)Why hotswap? Because opening a case to retrieve a failed drive is troublesome (case has to have access cleared to it, risk of pulling wrong drive on a running system, etc.)
On the subject of controllers: A good HBA (and SAS expander) are far better than using hardware raid controllers. Modern PCs have more than enough horsepower to push checksum calculations and it means that drives are portable between controllers.
I know about the cube cases out there such as Lian Li's D8000 - but afaik this doesn't allow for hotswap drives.