Test Setup - Software

With the variety of disk drive benchmarks available, we needed a means of comparing the true performance of the hard drives in real world applications. While we will continue to utilize HDTach and PCMark05 for comparative benchmarks our logical choice for application benchmarking is the Intel IPEAK Storage Performance Toolkit version 3. We originally started using this storage benchmark application in our Q2 2004 Desktop Hard Drive Comparison. The iPeak test can be designed to measure "pure" hard disk performance, and in this case we kept the host adapter consistent while varying the hard drive models. The idea is to measure the performance of individual hard drives with a consistent host adapter.

We utilize the IPEAK WinTrace32 program to record precise I/O operations when running real world benchmarks. We then utilize the IPEAK AnalyzeTrace program to review the disk trace file for integrity and ensure our trace files have properly captured the activities we required. Intel's RankDisk utility is used to play back the workload of all I/O operations that took place during the recording. RankDisk generates results in a mean service time in milliseconds format; in other words, it gives the average time that each drive took to fulfill each I/O operation. In order to make the data more understandable, we report the scores as an average number of I/O operations per second so that higher scores translate into better performance in all of our iPeak results. While these measurements will provide a score representing "pure" hard drive performance, the actual impact on the real world applications can and will be different.

Each drive is formatted before each test run and three tests are completed in order to ensure consistency in the benchmark results. The high and low scores are removed with the remaining median score representing our reported result. We utilize the NVIDIA nF4 SATA ports along with the NVIDIA IDE-SW driver to ensure consistency in our playback results when utilizing NCQ, TCQ, or RAID settings. Although we test NCQ capabilities, all of our reported results are generated with NCQ off unless otherwise noted.

Our iPeak tests represent a fairly extensive cross section of applications and usage patterns for both the general and enthusiast user. We will continually tailor these benchmarks with an eye towards the drive's intended usage and feature set when compared to similar drives. In essence, although we will reports results from our test suite for all drives, it is important to realize a drive designed for PVR duty will generate significantly different scores in our gaming benchmarks than a drive designed with gaming in mind such as the WD Raptor. This does not necessarily make the PVR drive a bad choice for those who capture and manipulate video while also gaming. Hopefully our comments in the results sections will offer proper guidance for making a purchasing decision in these situations. Our iPeak Test Suite consists of the following benchmarks.

VeriTest Business Winstone 2004: trace file of the entire test suite that includes applications such as Microsoft Office XP, WinZip 8.1, and Norton Antivirus 2003.

VeriTest Multimedia Content Creation 2004: trace file of the entire test suite that includes applications such as Adobe Photoshop 7.01, Macromedia Director MX 9.0, Microsoft Windows Media Encoder 9.0, Newtek Lightwave 3D 7.5b, and others.

AVG Antivirus 7.1.392: trace file of a complete antivirus scan on our test bed hard drive.

Microsoft Disk Defragmenter: trace file of the complete defragmentation process after the operating system and all applications were installed on our test bed hard drive.

WinRAR 3.51: trace file of creating a single compressed file consisting of 444 files in 10 different folders totaling 602MB. The test is split into the time it takes to compress the files and the time it takes to decompress the files.

File Transfer: individual trace files of transferring the Office Space DVD files to our source drive and transferring the files back to our test drive. The content being transferred consists of 29 files with a content size of 7.55GB.

AnyDVD 5.9.6: trace file of the time it takes to "rip" the Office Space DVD. We first copy the entire DVD over to our source drives, defragment the drive, and then measure the time it takes for AnyDVD to "rip" the contents to our test drive. While this is not ideal, it does remove the optical drive as a potential bottleneck during the extraction process and allows us to track the write performance of the drive.

Nero Recode 2: trace file of the time it takes to shrink the entire Office Space DVD that was extracted in the AnyDVD process into a single 4.5GB DVD image.

Game Installation: individual trace files of the time it takes to install Sims 2 and Battlefield 2. We copy each DVD to our secondary test drives, defragment the drive, and then install each game to our source drive.

Game Play: individual trace files that capture the startup and about 15 minutes of game play in each game. The Sims 2 trace file consists of the time it takes to select a pre-configured character, setup a university, downtown, business from each expansion pack (pre-loaded), and then visit each section before returning home. Our final trace file utilizes Battlefield 2 and we play the Daqing Oilfield map in both single and multiplayer mode.

Feature Comparison and Test Hardware Hard Disk Performance: HDTach / HD Tune
Comments Locked

18 Comments

View All Comments

  • semo - Monday, February 5, 2007 - link

    Gary, are you thinking of including some ssds (slc and mlc) in the mix for future comparisons. also, are you planning on doing a raid article (again with ssds too) and see if raid edition drives make a difference.
  • Gary Key - Monday, February 5, 2007 - link

    Hi,

    We will have a ssds roundup in March if the products are released on schedule. We will concentrate on SLC first as the MLC drive I do have is just terrible for general desktop usage. It was designed for industrial use and even I would not want to be a user at that workstation. ;) I am working on RAID article for March that will cover several chipsets and drives along with some new benchmarks.
  • oDii - Monday, February 5, 2007 - link

    Gary, would it be possible along side the various chipsets to see how Linux Software RAID performs (http://tldp.org/HOWTO/Software-RAID-HOWTO-5.html">http://tldp.org/HOWTO/Software-RAID-HOWTO-5.html or XFS)? It'd be great to see the results in context, as I haven't been able to find a complete and reliable source of results.
  • semo - Monday, February 5, 2007 - link

    thanks!
    the only reason i wanted to see an mlc drive in a roundup is to get an idea how bad they are but i get the picture now.

    i wonder if the faster response of the ssds compensate for their lower transfer rates and beat hdds in general usage. i guess we'll find out in march.
  • mostlyprudent - Monday, February 5, 2007 - link

    I should wait to see some numbers from the versions with 16MB cache sizes, but for me - this article reaffirms my choice of the Seagate 7200.10 320GB.
  • mjz - Monday, February 5, 2007 - link

    i'm amazed that the raptor didn't do so good.. why couldn't they just combine the 160 platter with the 10000 rpm
  • DrMrLordX - Tuesday, February 6, 2007 - link

    I kinda agree, though the newer 74 gig Raptor w/ 16 meg cache is supposedly faster than the 150 gig Raptor.

    Personally I'd rather see the 74 gig Raptor in there, but . . .
  • Gary Key - Tuesday, February 6, 2007 - link

    I will have a short performance update to include the 74GB 16MB cache Raptor tomorrow, not a full article but enough results to draw a conclusion.

Log in

Don't have an account? Sign up now