Yesterday Microsoft released the first public beta of the next version of Windows Home Server, currently going under the codename of Vail (or as we like to call it, WHS v2). WHS v2 has been something of a poorly kept secret, as word leaked out about its development as early as 2008. In more recent times an internal beta leaked out late last year, confirming that WHS v2 existed and giving everyone an idea of what Microsoft has in store for the next iteration of their fledgling home server OS.

As was the case with WHS v1, WHS v2 is having what we expect to be a protracted open beta period to squish bugs and to solicit feedback. This is very much a beta release by Microsoft’s definition (as opposed to Google’s definition) so it’s by no means production-worthy, but it’s to the point where it’s safe to throw it on a spare box or virtual machine and poke at it. Being based on Windows Server 2008 R2 (WHS v1 was based on Server 2003), the underlying OS is already complete, so what’s in flux are WHS-specific features.

 

We’ll cover WHS v2 more in depth once it’s closer to shipping, but there’s one interesting thing that caught our eye with WHS v2 that we’d like to talk about: Drive Extender.

Drive Extender v2: What’s New

Drive Extender was the biggest component of the secret sauce that made WHS unique from any other Microsoft OS. It was Drive Extender that abstracted the individual hard drives from the user so that the OS could present a single storage pool, and it was Drive Extender that enabled RAID-1 like file duplication on WHS v1. Drive Extender was also the most problematic component of WHS v1 however: it had to be partially rewritten for WHS Power Pack 1 after it was discovered that Drive Extender was leading to file corruption under certain situations. The new Drive Extender solved the corruption issues, but it also slightly changed the mechanism of how it worked, doing away with the “landing zone” concept in favor of writing files to their final destination in the first place.

For WHS v2, Microsoft has once again rewritten Drive Extender, this time giving it a full overhaul. Microsoft hasn’t released the full details on how Drive Extender v2 works, but we do know a few things that are worth discussing.

At the highest level, Drive Extender v1 was file based, sitting on top of the file system with the network shares going through it to access files. Hard drives in turn were NTFS formatted and contained complete files, with Drive Extender directing the network sharing service to the right drive through the use of NTFS tombstones and reparse points.

Drive Extender v2 is turning this upside down. The new Drive Extender is implemented below the file system, putting it between NTFS and the hard drives. With this change also comes a change in how data is written to disk – no longer are the drives NTFS formatted with the files spread among them, rather now they’re using a custom format that only WHS v2 can currently read. With Drive Extender v2 sitting between NTFS and the hard drives, anything that hits NTFS direct now receives full abstraction without the need to go through shared folders. In a nutshell, while Drive Extender v1 was file based, v2 is block based.

With the move to block based storage and a custom file system also comes a change in how data is stored on hard drives. In place of storing files on a plain NTFS partition, WHS v2 now uses a custom file system that stores data in 1GB chunks, with files and parts of files residing in these chunks. In the case of folder duplication, the appropriate chunks are duplicated instead of files/folders as they were on WHS v1.

Microsoft also threw in a bit more for data integrity, even on single-drive systems. Drive Extender v2 now keeps an ECC hash of each 512 byte hard drive sector (not to be confused with a chunk) in order to find and correct sector errors that on-drive ECC can’t correct – or worse, silently passes on. MS’s ECC mechanism can correct 1-bit and 2-bit errors, though we don’t know if it can detect larger errors. MS pegs the overhead at this at roughly 12% capacity, so our best guess is that they’re using a 64bit hash of each sector.

The Good & The Bad

All of these changes to Drive Extender will bring some significant changes to how WHS will operate. In terms of benefits, the biggest benefit is that from a high-level view WHS now operates almost exactly like NTFS. The NTFS volumes that Drive Extender v2 generates can be treated like any other NTFS volumes, enabling support for NTFS features such as Encrypted File System and NTFS compression that WHS v1 couldn’t handle. This also goes for application compatibility in general – Drive Extender v2 means that application compatibility is greatly improved for applications that want to interact directly with the drive pool, with this being one of Microsoft’s big goals for WHS v2.

Meanwhile at the low-level, using block based storage means that Drive Extender v2 doesn’t behave as if it was tacked on. Open files are no longer a problem for Drive Extender, allowing it to duplicate and move those files while they’re open, something Drive Extender v1 couldn’t do and which would lead to file conflicts. Speaking of duplication, it’s now instantaneous instead of requiring Drive Extender to periodically reorganize and copy files. The move to block based storage also means that WHS v2 data drives and their data can be integrated in to the storage pool without the need to move the data in to the pool before wiping and adding the drive, which means it’s going to be a great deal easier to rebuild a system after a system drive failure. Finally, thanks to chunks file size is no longer limited by the largest drive - a single file can span multiple drives by residing in chunks on multiple drives.

 

However this comes with some very notable downsides, some of which MS will have to deal with before launching WHS v2. The biggest downside here is that only WHS v2 can read MS’s new data drive file system – you can no longer pull a drive from a WHS and pull off files by reading it as a plain NTFS volume. This was one of WHS’s best recovery features, as a failing drive could be hooked up to a more liberal OS  (i.e. Linux) and have the data pulled off that way.

The other major downside to the new Drive Extender is that the chunking system means that a file isn’t necessarily stored on a single disk. We’ve already touched on the fact that excessively large files can be chopped up, but this theoretically applies to any file big enough that it can’t fit in to leftover space in an existing chunk. This in turn gives WHS v2 a very RAID-0 like weakness to drive failures, as a single drive failure could now take out many more files depending on how many files have their pieces on the failed drive. Our assumption is that Drive Extender v2 is designed to keep files whole and on the same disk if at all possible, but we won’t be able to confirm this until MS releases the full documentation on Drive Extender v2 closer to WHS v2’s launch.

First Thoughts

In discussing WHS v2 and Drive Extender v2, there’s one thing that keeps coming to mind: ZFS. ZFS is Sun/Oracle’s next-generation server file system that has been in continuing development now for a few years. ZFS is a block based file system for use in creating storage pools that features easy expandability and block hashing (among other features) making it not unlike Drive Extender v2. ZFS ultimately goes well beyond what Drive Extender v2 can do, but the fundamental feature set the two share is close enough that Drive Extender v2 can easily be classified as a subset of ZFS. Whether this is intentional or not, based on what we know thus far about Drive Extender v2, Microsoft has created what amounts to a ZFS-lite low-level file system with NTFS sitting on top of it. Make no mistake, this could prove to be a very, very cool feature, and is something we’re going to be keeping an eye on as WHS v2 approaches release.

In the meantime, Microsoft is still taking feature suggestions and bug reports on WHS v2 through Microsoft Connect. We don’t have any specific idea on how long WHS v2 will be in beta, but we’d bet on a Winter 2010 release similar to WHS v1 back in 2007, which means it would be in beta for at least a couple more months.

POST A COMMENT

52 Comments

View All Comments

  • gipper - Wednesday, April 28, 2010 - link

    Please please please let this fix the file conflicts issue.

    These corrupt files are KILLING me. Duplication is virtually worthless right now, and how in the heck can it NOT tell you WHICH physical disk is giving it trouble?
    Reply
  • clex - Wednesday, April 28, 2010 - link

    Supposedly its fixed in this version. You can even remove hard drives that have files open on them. Files open on another computer can now be moved around the drive pool. Reply
  • Dug - Wednesday, April 28, 2010 - link

    Try Disk Management Add-In. Reply
  • rrinker - Wednesday, April 28, 2010 - link

    I've had my system running for a year and a half, storing everything from software installes to movies and music, and have had not one problem with file corruption on duplicated folders - and I have EVERYTHING duplicated. I only log in to the box once in a while to check on disk space, otherwise it simply sits there and runs unattended on the floor alongside my desk. I used all WD Green drives in it, and with 5 drives it's quieter than the single Black 750GB drive in my desktop.
    The Disk Management add-in is probably the best thing ever, it displays all the stats on each drive, including SMART statistics so you can see drive activity and temperatures. The wireframe model is great when you roll your own and have 5 or 6 drives inside a case - it gives you an easy way to relate a specific device to the physical location inside the box.
    I'll say it again the new version can't come soon enough, I want to use 2TB drives so I don;t have to find a motherboard with 10+ SATA ports on it. It's going to take a while to copy all my data to a new box but starting over will be worth it to take advantage of AHCI mode and large drives, and the latest in low power processors (my existing server has an AMD 4050e dual core). I can't see how I would have been happy with a plain old NAS device in this role, I started with 1.5TB in this box and expanded it to over 4. Combined with a media player (I have a Popcorn Hour) it can't be beat. Stable, reliable, and nearly invisible.
    Reply
  • koshling - Wednesday, April 28, 2010 - link

    One thing this level of abstraction could provide is copy-on-write semantics allowing file hierrachy copies to be near instantanious (just duplicate block references) with block chnages only happening (splitting the modified copies) on subsequent writes. This is not only a lot faster buit alos a great deal more storage efficient since duplicated files that are NOT subsequently modified are only stored once. I wonder how this would play with Sandforce if the underlying storage pool is comprised of Sandforce SSDs - might eliminate a lot of redundancy Sandforce uses to extract its performance? Reply
  • -=Hulk=- - Wednesday, April 28, 2010 - link

    Do you thing that it is still possible to use a hardware raid 1 solution (like Intel chipset) instead of folder duplication?
    I find the folder duplication feature totally useless (for my usage), all my folders would have to be duplicated anyway and the negativ effect would be that the system partition still wouldn't be duplicated.

    That explains why I use Raid 1 with my WHS v1, and I would like to continue to use it with WHS v2, event if there seem to be the ability to backup the system partition twice a day if needed. But I still prefer a real time full mirroring of the HDDs, 2 backup a day of the system backup don't seem to be enough in my opinion.

    But the driver extender of WHS v2 works in a lower level than the file system NTFS, unlike the one in WHS v1.

    Do you thing that I will be able to use Raid 1 with Vail?
    Reply
  • davepermen - Wednesday, April 28, 2010 - link

    you should still not use raid. what if you want to expand your pool to 3, 4, 5 disks? of different sizes?

    the disk mode they do is for flexiblity. once you understand that, you never want to touch raid again.

    for the system disk, well, you can back it up now. i'll have to test that out soon. still having to think about a plan on how to test that beta :)
    Reply
  • -=Hulk=- - Thursday, April 29, 2010 - link

    "you should still not use raid. what if you want to expand your pool to 3, 4, 5 disks? of different sizes?"

    I don't need it. I prefer to buy new high capacity discs than having too many discs that increases the probability of HDD failure.

    I bought 2 1TB HDD for WHS v1 for 2 years, and there are still enough space left. With WHS v2 I will buy 2 2TB HDD which are not too expensiv and use the 2 old 1TB ones as external HDD for server backup.
    Reply
  • AMv8(1day) - Thursday, April 29, 2010 - link

    So I'll be honest, I'm more of a Network/hardware guy. I'm not familiar with alot of the nuances of file storage and server implementation but I was hoping to run the OS on a 32GB SSD and keep it separate from the Storage pool, keeping the critical OS functions relatively safe from drive failure and allowing all of the HDD's to go to sleep while the OS specific processes are running exclusively on the lower powered SSD. Will this be feasible with this new Drive Extender v2? As far as I know it wasn't possible on v1. Reply
  • -=Hulk=- - Thursday, April 29, 2010 - link

    It won't work with a 32Go SSD.

    The system partition has to be at least 160GB big.
    Reply

Log in

Don't have an account? Sign up now