It’s hard to say the last year has been anything but rough for Microsoft’s Windows division. Although we found Windows Vista favorable upon its launch last year after watching it go through an usually drawn-out development process, such a sentiment hasn’t been shared by Windows users as a whole. Windows XP proved to be every bit the competition for Vista that Microsoft could ever fear it would be, at the time as when Vista was having its own post-launch pains. It was a bad combination, making for a bad year for Microsoft’s efforts in pushing its first new desktop OS in 5 years. Microsoft was looking to make a solid case for why Vista is a worthwhile successor to XP in a market notorious for a resistance to change, and they failed to do this thanks to a failure in immature technology and an inability to get a consistent and convincing message out.

In the year since then you could make the argument that Microsoft’s marketing efforts still haven’t improved, but you would be hard pressed to make the same argument about Vista itself. Since its release an unfortunately large number of bugs and quirks have been discovered in Vista, which has kept Microsoft busy patching them over the year, while to their chagrin many consumers sit on the side watching. To Microsoft’s credit they’ve done a lot with Vista well before the first service pack, various patches including the reliability & compatibility packs released over the last year have solved many of the earliest complaints about Vista; it already performs better and is less quirky across the board now than when it launched. But it goes without saying that this hasn’t been enough to solve all of Vista’s problems, putting a lot of watchful eyes on Service Pack 1.

There is a saying among software development circles that businesses as a whole won’t touch a Microsoft product until the first service pack; they would prefer to wait until a product has been widely used and the biggest problems identified & solved. It’s cold but effective logic that also puts a great deal of pressure on Microsoft. No matter how good (or bad) a product is, half of their customers won’t bat an eye until there’s a service pack, making the first such pack just as important as the product launch itself in some ways. Complicating matters further with the Vista launch in particular is that Microsoft has tied Windows Server 2008 to the Vista kernel; getting Windows Server 2008 out the door means any and all Vista problems that would hinder server operation need to be eliminated. The result is that Service Pack 1 is a big deal for Microsoft, they need to show consumers that they can fix what still ails the OS, they need to show businesses that it’s now ready for them to use, and they need to show server administrators that the core technology is so good that a reliable server can be built off of it.

Furthermore, with the progression of technology in the last year the timing couldn’t be any more critical. The 4GB address space barrier for 32bit x86 is finally beginning to rear its head with more average computer uses; RAM prices have nosedived with 8GB of RAM going for as little as $160, resulting in a wide and very real need for a 64-bit operating system (and XP64 being a poor fit for consumers). Meanwhile PC OEMs are finally warming up to the Extensible Firmware Interface (EFI) and are ready to start building systems with it, meaning they too must move beyond XP. Even governments are finding they need to move to Vista as of late, as new encryption standards come in to play which only Vista supports.

The result of this is that many different groups have been watching SP1 far more intently than past service packs. With the final version of SP1 in hand, today we’ll be looking at what Microsoft is bringing to the table with Vista’s first service pack. With a combination of new features, bug fixes, and performance improvements, there’s a great deal to this service pack that we’ll be covering so let’s get started.

What’s Fixed In SP1

Bug fixes are a big part of any Microsoft service pack, but not just for the bugs being fixed specifically by the service pack. Microsoft has released numerous hotfixes since Vista launched, correcting a number of issues declared significant enough that they need to be fixed before the next service pack, but minor enough that they’re not worth a full deployment and the kind of massive regression testing that entails. The result is that there are a number hotfixes already out that can potentially fix specific issues certain users are having, but because they aren’t well-tested they’re instead well-hidden with only a small number of users with extreme problems usually getting their hands on any given hotfix. Now that a service pack has arrived, Microsoft has rolled up all of these hotfixes into the service pack, in essence approving them for wide release and full support.

Among the 24 pages(!) of hotfixes that have been rolled into Vista SP1 are favorites such as the virtual address space fix and a fix for a conflict with NVIDIA’s USB controller and >2GB of RAM. Other additions include fixes for ejecting iPods, a fix for HybridSLI/HybridCrossfire (which is why the launch of these technologies is tied to SP1), and a fix for AMD Barcelona processors causing system reboots during Windows installations. While we could rattle off the entire 24 page list of hotfixes, the important thing to note here is that there are a number of small issues that have been “fixed” prior to SP1 but are only now being widely corrected. We’re going to spend most of our time going over the biggest and most noticeable fixes in SP1, but please keep in mind there are many more things addressed in this service pack than what we’re looking at today or are listed in Microsoft’s consumer-level product literature.

Among the most significant fixes to Vista in SP1 is Microsoft's work on further refining the User Account Control (UAC) prompts of Vista. Even after already being scaled down between the betas and Vista’s launch, these prompts are still rather prolific at times. An adjustment to the folder creation is the most prominently touted of these fixes, with the number of folder creation prompts (when creating a folder in a protected location) falling from four to one. Microsoft doesn’t list any further reductions in UAC dialogs, but as far as anecdotal evidence is concerned it certainly feels like they’ve done a bit more than that. This won’t change the public perception of UAC (or Apple jokes on the subject), but any reduction is welcome and perhaps will stem the tide of Vista users who are completely turning off this critical system feature.

Another significant fix appearing in SP1 is a partial resolution to the conflict between the MultiMedia Class Scheduler Service and networking. As we’ve talked about the issue a bit before, the Vista audio stack is now in user space, which has lead to a change in how it operates. MMCSS boosts the priority of multimedia processes to real-time levels so that lower-priority processes can’t interrupt multimedia playback. During this time many other operations are interrupted or delayed so that they do not themselves interrupt the audio stack. One area that is dialed back involves the network interfaces, which are limited to 10k packets per second as a precaution.

For SP1 we were hoping for a complete overhaul of the MMCSS so that it ceased adversely affecting network performance, unfortunately what we’re getting is something about mid-way towards that. With SP1 it is now possible to control the amount of network throttling that MMCSS does, which means that throttling hasn’t been removed completely nor has it even been adjusted as far as the defaults are concerned. A quick test with Microsoft’s NTttcp tool shows the throttling level remains the same post-SP1 as it was pre-SP1 (roughly 70Mbps on a 1000Mb connection), which means SP1 will not be bringing any immediate relief. Furthermore there’s no GUI component (or real documentation) for this tweak, so users will be left to directly modifying the registry, a very uninviting situation.

What we do know is that this tweak only affects network receive performance, with a key apparently dictating the maximum percentage of the amount of network traffic allowed while the MMCSS is actively working. The key:

HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Multimedia\SystemProfile\NetworkThrottlingIndex

...defaults to 10 (for 10%) and can be adjusted to between 1 and 100, with the system requiring a reboot between adjustments. We did some quick testing with this key and were easily able to set it to 70%, which got us around 550Mbps of bandwidth through NTttcp, and we probably could have gone higher - especially on multi-core platforms.

Default throttling (top) and throttling with an index value of 70 (bottom)

While this is a solution to the MMCSS throttling issue, it’s not a good solution. The default value still makes for rather anemic performance on gigabit networks and the nature of the solution means that there is no single correct value to use to maximize network performance while not interrupting the audio stack; the highest value is highly dependent on processor performance. As a result power users wanting to correct this deficiency will have to a lot of experimenting on their own to find the highest value their system can tolerate without affecting multimedia playback. Had this value at least been auto-sensing we wouldn’t be so disappointed in Microsoft, but at the end of the day this isn’t a great solution. We’ll fully admit the problem will only affect a small number of users (those with gigabit networks who need high network performance while using multimedia applications), but then we’re exactly that kind of user.

For what it’s worth, we did inadvertently discover that the MMCSS throttling process doesn’t engage when audio streams are using APIs other than WaveOut and DirectSound (i.e. aren’t directly routed through the user-mode audio stack). OpenAL and ASIO do not trigger throttling, which means it’s possible to have both unthrottled networking and proper multimedia playback under Vista, as long as there’s hardware present that can deal with these APIs. This may very well be good news for Creative Labs in particular, whose DirectSound-to-OpenAL “Alchemy” wrapper can be used to have DirectSound applications routed to OpenAL instead and preventing throttling.

Moving on, SP1 also introduces a few interesting fixes for the user experience. For anyone who has a system running a 32-bit version of Windows and 4GB of RAM, they will be well aware that in 32-bit mode not all of that RAM can be addressed, and that Windows reports the amount that can be addressed accordingly. With Vista SP1 Windows will now be reporting the amount of RAM in the system, and not the amount that can be addressed. The advantage of this is that it will reduce the number of computer owners thinking something is wrong because Windows doesn’t “see” all of their RAM; on the other hand this is clearly disadvantageous because they will no longer be informed that Windows in fact isn’t using all of their RAM, nor will there be an easy way any longer to tell how much RAM it is capable of using.

Another user experience change with SP1 is that password hints are no longer optional when accounts are being created. It turns out that OEMs were complaining to Microsoft that users were forgetting their passwords and had no easy way to recover control of their computer since the Administrator account is no longer active by default, so Microsoft has done something about it. Password hints are now mandatory for all user accounts so that forgetful users are less likely to forget their passwords. How that will affect people that then forget what their hints mean remains to be seen.

What’s Fixed in SP1, Cont

Among all of the fixes in SP1, the biggest and most widely noticed will be the changes Microsoft has made to how Vista copies and moves files. There’s no two ways about it, Vista is slow at copying files - in fact frequently much slower than XP. Microsoft had a good reason for picking the methods they did for Vista but the payoff clearly hasn’t been worth it, so they’ve gone back to the drawing board and modified their file copy methods slightly to improve performance. Vista still won’t perform as well as XP (and we’ll get to why that is in a second), but with SP1 it’s definitely faster than it was under the original version of Vista.

The story of why Vista’s file copy speeds are slow so is a long and interesting one, with Microsoft’s tech blogging guru Mark Russinovich providing a particularly lengthy and descriptive explanation of the issue. If you’re curious as to what the specific details of the situation are, we encourage you to read the whole blog. For everyone else we’re going to take the liberty of paraphrasing and condensing it a bit, so that we can go over the changes to Vista without talking about things at such a low level.

According to Microsoft, there are three issues with the CopyFileEx method in XP that they wanted to correct with Vista.

1. XP’s buffered file network operations resulted in the file being cached no fewer than 3 times: twice on a client requesting data and once more at the host sending data. This wasted a lot of memory, particularly on the client.
2. Copying large files would eat up a great deal of memory when write operations couldn’t keep up with read operations (solid state drives in particular are prone to this) and the read data would be cached until it could finally be written
3. File copying was a synchronous action that couldn’t be pipelined, resulting in poor performance over high-latency, high-bandwidth links; this is one of the areas Microsoft was working to optimize performance under Vista, which included optimizations at the networking level.

Microsoft’s solution to these issues for Vista was to implement a new copying method that used more asynchronous I/O operations, and to stop buffering certain kinds of copy operations. This method does indeed fix the issues that Microsoft had with the XP copy method, but it also introduced new issues that caused Vista’s poor file copy speeds. The problems in particular are that asynchronous I/O can result in out of order write operations that require additional disk head seeks, and that unbuffered copy operations mean that a file is not in memory should it need to be immediately read again after being written (which can occur due to indexing, thumbnail generation, etc). Finally, and the reason that Microsoft believes is the root of most of the Vista complaints, Explorer under XP cheated a bit with file copying operations; it considered the job done once it had finished writing a file to the write cache. Vista meanwhile doesn't consider the job done until it is done writing the file to disk, so Vista will almost never “win”.

So what was changed in Vista SP1 to improve file copy performance? The biggest change is that file copy operations are once again cached most of the time, in effect a near complete reversal of the changes made for the initial version of Vista. Network copy operations are still not cached on the client side because of the double-buffering issue that caching induces. There are still a few differences between Vista and XP such as Vista’s support for larger file I/O operations that Microsoft believes will allow Vista to outperform XP, so it’s not quite a complete return to XP’s copying methods, but now Vista SP1 should be faster than the original Vista and XP much of the time. The only significant loser here are file operations over high-latency high-bandwidth links, as these changes effectively undo Vista’s optimizations for those situations; Microsoft has decided to instead make some additional changes at the network protocol level by adding some new features to SMB2 that deal with the above situation. File operations over such links between Vista clients or Vista and Server 2008 as a result will still be fast, while operations between Vista and older clients (Server 2003, XP, etc) will once again be slower like on XP.

Given the above information, we’ve benchmarked both pre-SP1 and post-SP1 Vista on an assortment of file copy operations to get an idea of what kind of a performance boost we’re looking at. We looked at copy operations with 3 kinds of data sets: a single large 4GB file, a single medium 700MB file, and a 300MB collection of roughly 2000 small files.

The results are not a massive improvement, but they’re also not unexpected. For our 4.5GB file, the copy time is about the same as the file is too large to be completely cached, so it benefits little from these changes. With our medium and small copy tests however we do see an improvement, with the medium copy showing the biggest improvement at 20% faster, and the small copy receives a much smaller but still measurable 7% improvement. With a return to caching however, the biggest improvements will be felt in situations where the file will be accessed immediately after copying. When we attempt to copy the small file collection immediately after a previous copy, the second copy takes only 6 seconds instead of 20 seconds since the files are still in the cache. Under the right situations all of these file I/O changes can create a big improvement in performance, and when we get to our full benchmarking suite we’ll see one such situation.

As for network copy performance, the improvements are still not massive, but interestingly enough they are greater than our local copy performance improvements. Our smallest improvement is with our single file transfer test between 2 SP1 machines at 15%, while our greatest improvement is seen with a transfer between an SP1 system and a Windows Server 2003 system at a whopping 50%. For anyone that has had to struggle with Vista’s poor networking performance (and who is not a victim of MMCSS throttling) SP1 looks to give a very sizable boost in performance. We do need to note however that these results are extremely variable on a system-to-system basis due to factors such as hard drive speed and the network controller used. Testing with other machines in the lab came up with numbers that were sometimes better and sometimes worse, so please don't take these results as universal.

Our final file system benchmark is for handling compressed folders, another sore point that became obvious with Vista after it shipped. Explorer’s speed with compressed folders was absolutely abysmal no matter how much sugar you coated the numbers with, so Microsoft has made some improvements to Explorer along with the aforementioned file I/O changes that will boost performance.

The great thing about benchmarks is that if your performance is bad enough, you can both improve your performance a great deal and still have terrible performance at the same time, which is exactly what happened. Compared to WinRAR, Explorer’s decompression speeds are still criminally slow; we have never been nor will start being amused with Explorer’s handling of compressed folders. When it comes to performance, anything is better than Explorer here. The silver lining here is that SP1 has improved WinRAR’s already fast performance by a further and unexpected 28%, making the argument to use anything but Explorer a very easy one.

Wrapping up our look at SP1’s biggest fixes, Microsoft has fixed a data corruption issue with NTFS-formatted removable disks. With the limitations of FAT32 and the implementation of reliable NTFS drivers on Mac OS X and most Linux distros, NTFS has become an increasingly popular cross-platform disk format to succeed FAT32. However, Vista had what we have been told is a moderately occurring data corruption issue with removable disks using NTFS, which made such disks unsafe to use under Vista. SP1 fixes this, though Microsoft is still passively recommending against NTFS for external disks since NTFS was never designed for this use. Instead they’re recommending exFAT, which we’ll get to in a bit.

An installation issue with Vista x64 and certain chipsets has also been fixed in SP1. One of the disk controller drivers in the Windows installer cannot properly handle disk controllers that only support 32-bit DMA, resulting in a BSOD when the controller was requested to do an operation in a memory area beyond the 32-bit limit. Previously the solution was to install Vista x64 with 2GB (or less) of RAM so that it never did this, but now disks with SP1 integrated will not encounter this issue at all. This primarily affects only users of certain NVIDIA motherboard chipsets.

Finally, Microsoft has taken a bite out of Vista piracy with a surprising level of bluntness. SP1 fixes two specific vulnerabilities that allowed Vista to be pirated: the OEM BIOS exploit, and the Grace Timer exploit. It’s notable that Microsoft specifically named the exploits they were fixing, where in the past they simply have made vague references to fixing exploits that allow Windows piracy. Given Microsoft’s strong anti-piracy focus for Windows Vista, we’re a bit surprised they didn’t patch out these exploits sooner. It’s worth noting though that for illegitimate copies of Vista, SP1 does away with Vista’s “reduced functionality” mode; now it leaves all functionality enabled but repeatedly alerts/harasses the user about their Vista installing being unactivated or illegitimate.

What’s New In SP1

Traditionally Microsoft has focused nearly exclusively on fixes in their service packs, with new features being few and definitely not the focus of a service pack - new features instead usually come with a different OS. When SP2 for Windows XP was released in 2004, it broke this mold with a highly atypical number of new and long-needed features to go along with the fixes it integrated. Although at the time Microsoft called it an exception to the rule, Vista SP1 makes for a tradition of exceptions, bringing a large number of new features to Vista.

For new features, we’ll start with EFI. Vista x64 was previously scheduled to get Extensible Firmware Interface (EFI) support, but this was pulled before the launch of Vista for reasons that were never made clear. Microsoft does have a working EFI implementation for the Itanium versions of Windows, so it was not a case of them being completely unprepared. With SP1 having been released we finally have a good hunch what this reason was: it appears Microsoft was waiting for the Unified EFI Forum to complete their 2.1 specification.  Microsoft’s previous implementations were for Intel’s EFI implementation prior to Intel releasing the specification to the UEFI Forum, and while the first UEFI specification was completed in early 2006, the Forum made a number of small but significant changes for the 2.1 specification which didn’t come until early 2007.

The end result is that the long-promised support for EFI (or rather UEFI; the original EFI 1.xx isn’t supported due to issues with x64) is finally here. As was originally going to be the case, UEFI support is limited to the x64 editions of Vista. Microsoft continues to justify this by claiming that EFI support for 32-bit x86 systems is a dead end, an argument that is particularly convincing a year and a half later now that systems are finally shipping with 4GB of RAM and need to/should be run in 64-bit mode anyhow.

With that said this change won’t make much of an immediate difference for Vista, but it finally gives PC manufacturers the ability to use UEFI if they desire, without having to resort to BIOS compatibility modules for Windows. We’re still waiting for someone besides Apple to start shipping consumer machines (or motherboards) with UEFI support, so this will be an issue we’ll pick up another day. (Ed: We did see a few demonstrations of UEFI boards at CES, though they're not yet publicly available.) For now we’re still looking forward to what motherboard manufacturers can do when freed from the ancient 16-bit real-mode for the startup/configuration abilities, along with the new features like GUID Partition Tables that offer a nearly unlimited number of partitions and better partition resizing.

With the addition of UEFI support, Microsoft has also made a few tweaks to the Windows pre-installation environment that should be more immediately useful. For anyone that has attempted to install Vista with a disc containing both the x86 and x64 versions of the operating system, they will have first-hand experience with the fact that two pre-installation environments were required – one for each version of the OS as an environment could not install the other version of the OS. That experience has been unified somewhat with SP1; now the x86 environment can install the x64 version of the OS (but not the other way around, interestingly enough). This effectively fixes one of the more annoying quirks in the Vista installation process, although the combined size of both the x86 and x64 installers means such disks still aren’t the default since their contents can’t fit on a single-layer DVD. For now Microsoft is targeting this towards IT administrators who roll their own custom installer images and who now will only need one image no matter what their machine is (x86, x64, or x64-UEFI).

AMD’s graphics division is also getting a pick-me-up with SP1, with the inclusion of Direct3D 10.1 support. AMD’s HD3000 series cards are still the only cards to support D3D 10.1, but this has mattered little since D3D 10.1 wasn’t out at the time that AMD released those cards. This allows AMD to push the issue harder although we’re not sure it will make much of a difference. Given the slow adaptation of DirectX/Direct3D 10 by game developers, we haven’t seen any real momentum towards D3D 10.1. Developers may simply skip Direct3D 10.1 and go for Direct3D 11 when it is finally released, otherwise sticking with 10.0 for the time being. (Ed: We've heard from Microsoft and several game developers that DX10.1 is not a major update and that they will do exactly that.)

Hotpatching support has also finally been added to Vista, which like UEFI is another one of those features that was on the drawing board at one point but disappeared before Vista was released. The lack of hotpatching support, otherwise known as the ability to patch running software without a reboot, has long been an irksome issue with Windows. As Microsoft has implemented it, this support is limited to Windows components (as opposed to any dreams of driver hotpatching), and we’re eager to see some patches for Vista SP1 to see this feature in action and to judge whether Microsoft really is able to reduce the amount of reboots required in patching.

Also new to SP1 are a few security related APIs for application developer use. The first API is for Data Execution Prevention (DEP, aka the NX/XD bit), a buffer-overflow prevention feature that was introduced with XP SP2. By default DEP is only enabled for certain Microsoft services because of its unpredictable performance with applications not built and tested against it. With the addition of this API, developers will be able to control how DEP functions, so that if their code isn’t completely DEP-safe, they may disable certain parts of DEP for their specific application, allowing some protection from DEP without the need to rewrite the offending code or require that DEP be disabled for that program entirely. This is effectively a precursor towards Windows being globally DEP enabled at some later point.

The second security API is for security software vendors, some of whom were caught off guard by Vista x64’s Kernel Patch Protection feature. Certain security/anti-virus software patches the Windows kernel in order to enact their defensive operations, and with KPP this was prevented. The issue turned into a big enough political quagmire that the European Commission was looking in to the matter as an anticompetitive action. As a result Microsoft has developed an API to allow applications to exert some control over KPP and allow those (and only those) applications to patch the kernel. Allowing any patching seems like a poor idea that goes against the goals and security offered by KPP, but this is an issue that has long since been decided on, and the vendors requesting the ability to continue patching the kernel have won out.

Rounding out the major additions to Vista SP1 are items to support new technology. The more pressing of these is full support for 802.11n Draft 2.0 wireless networking, which in spite of not being a final version of the 802.11n standard has quickly become a de-facto standard. While it is possible for a pre-SP1 machine to use 802.11n, it requires an additional level of work by the hardware developer to write more driver code and applications to compensate for the lack of native support - the OS has such support for 802.11a/b/g, thus handling most of the work. In effect SP1 brings 802.11n support to the same level as a/b/g.

Finally we come to exFAT, the next-generation successor to the ubiquitous FAT32 file system. For anyone that has used FAT32 in recent times on a large drive, you should be familiar with its limitations in terms of files allowed in a single directory, a lack of security permissions/access control lists, and a particularly harsh 4GB limit for file sizes. The last two items in particular have been making FAT32 more difficult to use as file sizes continue to increase, and the move to Windows XP gave home users real file system security through a file system with ACL support (NTFS). exFAT in turn is designed to be FAT32’s successor, implementing a modern but still light file system design that supports all of these missing features (although Vista SP1 doesn’t appear to support ACLs, even though it’s part of the standard).

At this point in time exFAT exists in an odd space between FAT32 and NTFS that makes it hard to determine if Microsoft is going give exFAT a reasonable foothold. With the continuing perfection of NTFS drivers for non-Windows operating systems and Microsoft’s own fixes to NTFS removable disk support in SP1, NTFS has been slowly becoming the de-facto standard file system for cross-OS disk access, and like exFAT NTFS is a modern file system that doesn’t suffer from the problems posed by FAT32. Furthermore Microsoft has been successful in securing patents for FAT32 (a standard that was previously treated as an open one), making some groups leery of exFAT. exFAT does have an advantage over NTFS thanks to being a lighter weight file system. It's easier to deal with exFAT on devices with limited processing power and memory, and exFAT possesses a much smaller data structure overhead (we measured NTFS at 30MB vs. <1MB for exFAT on a 512MB flash drive), but this may not be enough.

exFAT as the common cross-OS file system seems unlikely at this point (as a result Microsoft is wisely not targeting it towards this use), so what support it does pick up will be limited to mobile devices. But how many mobile devices are in immediate need of ACLs? Or support for files over 4GB? There’s a somewhat convincing argument for using exFAT with digital picture frames if you can gather a large enough number of photos (roughly sixty-five thousand), but that’s the extent of "good" reasons to use exFAT at the moment. We’re not convinced Microsoft is going to see much use of exFAT outside of Windows Mobile 6 devices given the high degree of overlap with NTFS; if the time comes for mobile devices where FAT32 is too little, they may very well switch to NTFS due to the much wider base of support.

The Test

Along with testing individual aspects of Vista for specific performance fixes, we have also run a subset of our usual performance benchmarks to see if there are any other notable performance improvements - or alternatively if the performance improvements contained in Vista SP1 will spill out in to more generalized improvements in performance. To that end we’ve run tests on our benchmark rig with both a fully patched installation of Vista RTM/SP0, and again with SP1. Due to time constraints and the fact that we are using Vista x64, we did not include XP benchmarks at this time. With XP SP3 right around the corner, we’ll add XP into the mix soon enough.

Vista vs. Vista SP1

We’ll start with our Futuremark benchmark applications, 3DMark 2006 and PCMark Vantage. 3DMark in spite of being a graphics benchmark is sensitive enough to pick up on any changes in CPU or GPU performance (in this case any optimizations that reduce overhead), while PCMark Vantage is a full suite of benchmarks to measure overall system performance. It’s also one of the few benchmarks that we’re using that has a 64-bit mode.

Starting with 3DMark, the change in performance is effective imperceptible at 0.2%, well within experimental variance. This just goes to show there haven’t been any changes in overhead. PCMark however is far more interesting; here we get a full 7% score increase, indicating that SP1’s effects are felt outside of our earlier benchmarks. Drilling down ino PCMark’s subscores, we found that the higher score is a result of improvements in data compression scores, data encryption scores, and searching in Windows Mail, all of which can be attributed to improvements in file I/O performance. All other subscores are virtually unchanged.

Moving on to our application specific benchmarks, we have our DivX encoding test, our iTunes/LAME MP3 encoding test, and the Retouch Artists speed test for Photoshop. DivX stresses I/O somewhat, while the rest of the tests are largely memory and CPU-bound.

Here we see no notable changes in performance moving to SP1. All of the application tests come back with virtually identical scores.

Finally we have our gaming tests. Games tend to be good a great way to stress all the components in a system, so this should give us a better idea of how far improvements in Vista’s file I/O system stretch. For our games we have the RTSes World in Conflict and Company of Heroes, and the FPSes Crysis and Unreal Tournament 3.

In spite of the more rounded nature of gaming tests compared to our application tests, the results are the same with no perceivable improvement in performance. At this point it’s clear that what performance improvements Vista does offer are limited to a handful of situations where we are specifically file or network I/O bound. What this means for any given application is that it is unlikely to see a performance improvement due to SP1.

We also ran some quick testing with startup and shutdown times to see if Vista improved performance there at all; there were a couple of hotfixes in SP1 that dealt with these matters.

While shutdown time doesn’t see any real performance improvements, we are surprised to see an improvement in startup time by several seconds. Vista is not notably slow to start up in the first place, so we weren’t expecting much improvement if there was to be any at all. Shaving off 5 seconds for a 14% improvement in startup time (getting startup below 30 seconds altogether) is a pleasant surprise.

Observations and Closing Thoughts

As far as the Vista user experience is concerned, users shouldn’t expect any significant changes with SP1. In this respect Vista SP1 is much like any other Windows service pack, rather than being another XP SP2. To that extent if you threw a pre-SP1 system and a post-SP1 system in front of us, we’d need to do some low-level benchmarking to identify which one was using SP1. In day-to-day use, the difference is not obvious outside of the specific improvements we’ve talked about.

For those curious about how long the SP1 installation process takes, Microsoft has said it will take anywhere between 20 minutes to over an hour. Some of this boils down to simple hard drive performance, with slower drives taking longer to update all of the files SP1 patches. Given our own installation efforts, we suspect that there are other factors that are non-obvious - in other words, your mileage may vary. In general Vista x64 will take longer to patch than Vista x32 due to the additional files that need to be patched under Vista x64 (e.g. there are a number of files and libraries that come in 32-bit and 64-bit versions). On our official test system we clocked Vista x64 at 33.5 minutes to install from start to finish, while a laptop took just shy of an hour. You’ll definitely want to go find something else to do for a bit while Vista is patching, and if you're running an ultraportable laptop with a 1.8" hard drive you will very likely break the one hour threshold.

One thing that is unfortunate for Microsoft with SP1 is that there is a good chance that system performance immediately following the patching process will be lower than it was prior to patching. As part of the installation process the SuperFetch and ReadyBoost subsystems are purged of all caches and learned behaviors, effectively reverting a patched system to that of a brand-new untrained system. Vista does not take long to retrain itself, and Microsoft notes the process can take a couple of days (we were back to perceived normal within a day), but nevertheless a lot of people are going to be thrown off by things such as longer application load times immediately following the patch.

Finally, coming into SP1 we heard some concerns about application and driver compatibility. While we cannot test everything, we have not run into any new issues with SP1. We have heard within the last day that a small number of systems are having an issue with one of the SP1 pre-patches (patches that are required prior to installing SP1) causing an infinite reboot sequence, but we have not experienced this first hand, nor do we have an accurate idea of how large the affected “small” group of users is, given the echo chamber effect on the Internet. We cannot recall a Windows service pack that didn’t break at least a handful of Windows installations, so this could simply be par for the course; it’s hard to say at this point.

At the end of the day, we don’t have much of anything bad to say about SP1 outside of the “fix” for displaying the amount of installed memory on 32-bit systems, so our recommendation is that all Vista users to install SP1 once it becomes available to the public at large. It won’t knock most people’s socks off, but the file and network performance improvements are long overdue and will be noticeable for most users. Ultimately, any user who has felt slighted by the poor copy performance of Vista will find relief in SP1, as will anyone whose pet-issue has specifically been fixed in Vista SP1. Anyone else who didn’t like Vista for other reasons will be no more impressed by SP1 than they were by the original version; there are a few quirks that should have been resolved in SP1 that were not.

Compared to where we were a year ago, our general recommendation for Vista is unchanged. We are however impressed with the progress of the x64 versions of Vista over the past year, after feeling like it was lagging behind Vista x86 from beta up through the release version of Vista. Vista x64 is now clearly on par with Vista x86 and we have no concerns about its compatibility or performance. There are still good reasons to stick with Vista x86, such as compatibility with specific applications and Vista x64’s higher memory usage due to WoW64, but these are the only reasons. A year ago we recommended using Vista x86 unless you specifically needed Vista x64; now we’re comfortable making the opposite recommendation of running Vista x64 unless you have a specific reason to stick with Vista x86.

Finally, for those Windows users still sticking with XP, they too will be getting Microsoft's long-overdue XP SP3 in the very near future. We’ll be bringing a review of that to you as soon as it goes gold later this quarter, along with a fully up to date performance comparison between Vista and XP to better illustrate what little gap remains between the two operating systems. The list of changes isn’t nearly as far-reaching as Vista SP1, but there are a couple of interesting items on the list. (Ed: It will also be nice to not install over 100 patches/updates/etc. after a clean XP SP2 install.) Stay tuned for that in the coming weeks.