Yesterday Microsoft released the first public beta of the next version of Windows Home Server, currently going under the codename of Vail (or as we like to call it, WHS v2). WHS v2 has been something of a poorly kept secret, as word leaked out about its development as early as 2008. In more recent times an internal beta leaked out late last year, confirming that WHS v2 existed and giving everyone an idea of what Microsoft has in store for the next iteration of their fledgling home server OS.

As was the case with WHS v1, WHS v2 is having what we expect to be a protracted open beta period to squish bugs and to solicit feedback. This is very much a beta release by Microsoft’s definition (as opposed to Google’s definition) so it’s by no means production-worthy, but it’s to the point where it’s safe to throw it on a spare box or virtual machine and poke at it. Being based on Windows Server 2008 R2 (WHS v1 was based on Server 2003), the underlying OS is already complete, so what’s in flux are WHS-specific features.

 

We’ll cover WHS v2 more in depth once it’s closer to shipping, but there’s one interesting thing that caught our eye with WHS v2 that we’d like to talk about: Drive Extender.

Drive Extender v2: What’s New

Drive Extender was the biggest component of the secret sauce that made WHS unique from any other Microsoft OS. It was Drive Extender that abstracted the individual hard drives from the user so that the OS could present a single storage pool, and it was Drive Extender that enabled RAID-1 like file duplication on WHS v1. Drive Extender was also the most problematic component of WHS v1 however: it had to be partially rewritten for WHS Power Pack 1 after it was discovered that Drive Extender was leading to file corruption under certain situations. The new Drive Extender solved the corruption issues, but it also slightly changed the mechanism of how it worked, doing away with the “landing zone” concept in favor of writing files to their final destination in the first place.

For WHS v2, Microsoft has once again rewritten Drive Extender, this time giving it a full overhaul. Microsoft hasn’t released the full details on how Drive Extender v2 works, but we do know a few things that are worth discussing.

At the highest level, Drive Extender v1 was file based, sitting on top of the file system with the network shares going through it to access files. Hard drives in turn were NTFS formatted and contained complete files, with Drive Extender directing the network sharing service to the right drive through the use of NTFS tombstones and reparse points.

Drive Extender v2 is turning this upside down. The new Drive Extender is implemented below the file system, putting it between NTFS and the hard drives. With this change also comes a change in how data is written to disk – no longer are the drives NTFS formatted with the files spread among them, rather now they’re using a custom format that only WHS v2 can currently read. With Drive Extender v2 sitting between NTFS and the hard drives, anything that hits NTFS direct now receives full abstraction without the need to go through shared folders. In a nutshell, while Drive Extender v1 was file based, v2 is block based.

With the move to block based storage and a custom file system also comes a change in how data is stored on hard drives. In place of storing files on a plain NTFS partition, WHS v2 now uses a custom file system that stores data in 1GB chunks, with files and parts of files residing in these chunks. In the case of folder duplication, the appropriate chunks are duplicated instead of files/folders as they were on WHS v1.

Microsoft also threw in a bit more for data integrity, even on single-drive systems. Drive Extender v2 now keeps an ECC hash of each 512 byte hard drive sector (not to be confused with a chunk) in order to find and correct sector errors that on-drive ECC can’t correct – or worse, silently passes on. MS’s ECC mechanism can correct 1-bit and 2-bit errors, though we don’t know if it can detect larger errors. MS pegs the overhead at this at roughly 12% capacity, so our best guess is that they’re using a 64bit hash of each sector.

The Good & The Bad

All of these changes to Drive Extender will bring some significant changes to how WHS will operate. In terms of benefits, the biggest benefit is that from a high-level view WHS now operates almost exactly like NTFS. The NTFS volumes that Drive Extender v2 generates can be treated like any other NTFS volumes, enabling support for NTFS features such as Encrypted File System and NTFS compression that WHS v1 couldn’t handle. This also goes for application compatibility in general – Drive Extender v2 means that application compatibility is greatly improved for applications that want to interact directly with the drive pool, with this being one of Microsoft’s big goals for WHS v2.

Meanwhile at the low-level, using block based storage means that Drive Extender v2 doesn’t behave as if it was tacked on. Open files are no longer a problem for Drive Extender, allowing it to duplicate and move those files while they’re open, something Drive Extender v1 couldn’t do and which would lead to file conflicts. Speaking of duplication, it’s now instantaneous instead of requiring Drive Extender to periodically reorganize and copy files. The move to block based storage also means that WHS v2 data drives and their data can be integrated in to the storage pool without the need to move the data in to the pool before wiping and adding the drive, which means it’s going to be a great deal easier to rebuild a system after a system drive failure. Finally, thanks to chunks file size is no longer limited by the largest drive - a single file can span multiple drives by residing in chunks on multiple drives.

 

However this comes with some very notable downsides, some of which MS will have to deal with before launching WHS v2. The biggest downside here is that only WHS v2 can read MS’s new data drive file system – you can no longer pull a drive from a WHS and pull off files by reading it as a plain NTFS volume. This was one of WHS’s best recovery features, as a failing drive could be hooked up to a more liberal OS  (i.e. Linux) and have the data pulled off that way.

The other major downside to the new Drive Extender is that the chunking system means that a file isn’t necessarily stored on a single disk. We’ve already touched on the fact that excessively large files can be chopped up, but this theoretically applies to any file big enough that it can’t fit in to leftover space in an existing chunk. This in turn gives WHS v2 a very RAID-0 like weakness to drive failures, as a single drive failure could now take out many more files depending on how many files have their pieces on the failed drive. Our assumption is that Drive Extender v2 is designed to keep files whole and on the same disk if at all possible, but we won’t be able to confirm this until MS releases the full documentation on Drive Extender v2 closer to WHS v2’s launch.

First Thoughts

In discussing WHS v2 and Drive Extender v2, there’s one thing that keeps coming to mind: ZFS. ZFS is Sun/Oracle’s next-generation server file system that has been in continuing development now for a few years. ZFS is a block based file system for use in creating storage pools that features easy expandability and block hashing (among other features) making it not unlike Drive Extender v2. ZFS ultimately goes well beyond what Drive Extender v2 can do, but the fundamental feature set the two share is close enough that Drive Extender v2 can easily be classified as a subset of ZFS. Whether this is intentional or not, based on what we know thus far about Drive Extender v2, Microsoft has created what amounts to a ZFS-lite low-level file system with NTFS sitting on top of it. Make no mistake, this could prove to be a very, very cool feature, and is something we’re going to be keeping an eye on as WHS v2 approaches release.

In the meantime, Microsoft is still taking feature suggestions and bug reports on WHS v2 through Microsoft Connect. We don’t have any specific idea on how long WHS v2 will be in beta, but we’d bet on a Winter 2010 release similar to WHS v1 back in 2007, which means it would be in beta for at least a couple more months.

POST A COMMENT

52 Comments

View All Comments

  • Ryan Smith - Wednesday, April 28, 2010 - link

    Supposedly it's a scalability issue that they hope to lessen for RTM. I don't think it's going to be a settled issue until then. Reply
  • clex - Wednesday, April 28, 2010 - link

    Its just a recommendation for people using the beta to be aware that it has only been tested internally with 16 drives totaling 16TB. It has been mentioned by the WHS team that errors begin to happen after that. They said that they are trying to fix this "bug" before WHS v2 ships. Reply
  • MadMan007 - Wednesday, April 28, 2010 - link

    Great, another WHS drive extender bug...'hopefully' solved before RTM. I'm not too sure the downsides outweigh the upsides here. 12% lost space is nothing to sneeze at in addition to the other negatives. Reply
  • funkyd99 - Wednesday, April 28, 2010 - link

    ..but the addition of error correction is huge in my opinion, assuming it works correctly. Take the following situation I had to deal with last year:

    1. WHS reports a disk as unhealthy. Repairing disk returns it to healthy. Running a manual chkdsk /r files a handful of bad sectors, but I pass it off as a one-time error.

    2. A week later and the same problem crops up. Repeat step one and also run disk through manufacturer's diagnostic utility, which reports disk as "healthy". I return the disk to the server and examine the shared folders.

    3. Notice a decent amount of corrupted files on the server. Replace drive and restore all files from a backup.

    I can only assume WHS was overwriting valid files with versions that were "fixed" by chkdsk. I'd gladly take a 12% storage hit if I don't have to worry about situations like this cropping up again.
    Reply
  • gipper - Wednesday, April 28, 2010 - link

    I'm dealing with this right now. However, I have four drives and can't find the culprit. File corruption is not a feature I'd expect from a system that includes "duplication" and "redundant storage". Plus, whs has no way of identifying the offending disk to you. Reply
  • gipper - Wednesday, April 28, 2010 - link

    I'm dealing with this right now. However, I have four drives and can't find the culprit. File corruption is not a feature I'd expect from a system that includes "duplication" and "redundant storage". Plus, whs has no way of identifying the offending disk to you. Reply
  • funkyd99 - Wednesday, April 28, 2010 - link

    I thought the offending disk was shown in the Server Storage tab? Or are you saying you aren't sure what physical drive has the errors? If that's the case, you can download the Disk Management plugin (you may still be able to find the older free version, but it's worth $10 IMO), which will tell you what channel the offending drive is connected to. Reply
  • Brian Klug - Wednesday, April 28, 2010 - link

    I love the concept of WHS, I really do. In fact, it's impossible to argue that this kind of model isn't the future, simply because as you add work devices (we're up to a desktop, laptop, netbook and possibly an iPad now), you need to centralize storage somewhere common to preserve sanity. But until Microsoft can find a way to span files across volumes in a pseudo-RAID5 fashion with similar overhead and data integrity, I can't see myself using it.

    I mean, that's the real inherent danger that they're creating here; more drives increases your chance of drive failure resulting in data loss. Creating huge volumes that span across more than 3 drives without using RAID5 just scares me. I'm unclear about how the data duplication works with the new drive extender - though it'd seem that breaking files up into blocks would make it much easier to say, take all your data and back it up redundantly across unused space on the other drives. Microsoft argues that WHS is still faster than their own software RAID5; any validity those claims? In practice, I always see hardware limitations before software on my server running software RAID.

    Full disclosure here is that I currently run a home server of my own with 2 large RAID5 arrays (4x750GB), (4x1.5 TB). Although I'd love to switch (just because frankly I don't need a fully featured Windows Server install with all the overhead), I feel like power users that are going to want WHS really demand that kind of solid data duplication.

    I feel like my views on WHS might be outdated; does V2 address those concerns?
    Reply
  • clex - Wednesday, April 28, 2010 - link

    The differences between v1 and v2 are awesome. I've been running a WHS since it was released and love it (8 HDD totaling 6 TB). The new web interface is fantastic. When more people realize that they need this and how easy it is to use and administrate, WHS will no longer be a "best kept secret." Reply
  • davepermen - Wednesday, April 28, 2010 - link

    Indeed.

    Hope they find some way to market it better. maybe starting with a more homefriendly name? server doesn't fit. media hub maybe (fitting the new win7mobile hubs?), home center, or what ever.

    sadly media center is gone, which would actually fit quite well :)
    Reply

Log in

Don't have an account? Sign up now