Promise Pegasus R6 & Mac Thunderbolt Review
by Anand Lal Shimpi on July 8, 2011 2:01 AM ESTThe Pegasus: Performance
A single 2TB Hitachi Deskstar 7K3000 is good for sequential transfer rates of up to ~150MB/s. With six in a RAID-5 configuration, we should be able to easily hit several Gbps in bandwidth to the Pegasus R6. The problem is, there's no single drive source that can come close to delivering that sort of bandwidth.
Apple sent over a 15-inch MacBook Pro with a 256GB Apple SSD. This was the first MacBook Pro I've ever tested with Apple's own SSD, so I was excited to give it a try. The model number implies a Toshiba controller and I'll get to its performance characteristics in a separate article. But as a relatively modern 3Gbps SSD, this drive should be good for roughly 200MB/s. Copying a large video file from the SSD to the Pegasus R6 over Thunderbolt proved this to be true:
Apple's SSD maxed out at 224MB/s to the Thunderbolt array, likely the peak sequential read speed from the SSD itself. Average performance was around 209MB/s.
That's a peak of nearly 1.8Gbps and we've still got 8.2Gbps left upstream on the PCIe channel. I needed another option.
Without a second Thunderbolt source to copy to the array at closer to the interface's max speed, we had to generate data. I turned to Iometer to perform a 2MB sequential access across the first 1TB of the Pegasus R6's RAID-5 array. I ran the test for 5 minutes, the results are below:
| Promise Pegasus R6 12TB (10TB RAID-5) Performance | ||||||
| Sequential Read | Sequential Write | 4KB Random Read (QD16) | 4KB Random Write (QD16) | |||
| Promise Pegasus R6 (RAID-5) | 673.7 MB/s | 683.9 MB/s | 1.24 MB/s | 0.98 MB/s | ||
The best performance I saw was 683.9MB/s from our sequential write test, or 5471Mbps. Note that I played with higher queue depths but couldn't get beyond these numbers on the stock configuration. Obviously these are hard drives so random performance is pretty disappointing.
That's best case sequential performance, what about worst case? To find out I wrote a single 10TB file across the entire RAID-5 array then had Iometer measure read/write performance to that file in the last 1TB of the array's capacity:
| Promise Pegasus R6 12TB (10TB RAID-5) Performance | ||||||
| Sequential Read (Beginning) | Sequential Write (Beginning) | Sequential Read (End) | Sequential Write (End) | |||
| Promise Pegasus R6 (RAID-5) | 673.7 MB/s | 683.9 MB/s | 422.7 MB/s | 463.0 MB/s | ||
Minimum sequential read performance dropped to 422MB/s or 3.3Gbps. This is of course the downside to any platter based storage array. Performance on outer tracks is much better than on the inner tracks, so the more you have written to the drive the slower subsequent writes will be.
At over 5Gbps we're getting decent performance but I still wanted to see how far I could push the interface. I deleted the RAID-5 array and created a 12TB RAID-0 array. I ran the same tests as above:
| Promise Pegasus R6 12TB (10TB RAID-5) Performance | ||||||
| Sequential Read | Sequential Write | 4KB Random Read (QD16) | 4KB Random Write (QD16) | |||
| Promise Pegasus R6 (RAID-5) | 673.7 MB/s | 683.9 MB/s | 1.24 MB/s | 0.98 MB/s | ||
| Promise Pegasus R6 (RAID-0) | 782.2 MB/s | 757.8 MB/s | 1.27 MB/s | 5.86 MB/s | ||
Sequential read performance jumped up to 782MB/s or 6257Mbps. We're now operating at just over 60% of the peak theoretical performance of a single upstream Thunderbolt channel. For a HDD based drive array, this is likely the best we'll get.
To see how far we could push things I pulled out all six drives and swapped in four SF-2281 based SSDs. To really test the limits of the interface I created a 4-drive RAID-0 array sized at only 25GB. This would keep drive performance as high as possible and reduce the time required to fill and test the drives.
Unlike the hard drive based arrays, I had to take the queue depth up to 16 in order to get peak performance out of these SSDs. The chart below shows all of my performance attempts:
With highly compressible data, I managed to get just over 1000MB/s (8021Mbps to be exact) to the 4-drive SF-2281 Pegasus R6. Note that this isn't a shipping configuration, but it does show us the limits of the platform. I'm not entirely convinced that we're limited by Thunderbolt here either - it could very well be the Pegasus' internal controller that's limiting performance. Until we get some other Thunderbolt RAID devices in house it's impossible to tell but at around 8Gbps, this is clearly an interface that has legs.
Facebook
Twitter
RSS
88 Comments
View All Comments
etamin - Sunday, July 10, 2011 - link
hmm, I think I'm missing something here. Are you saying that the new MBPs have 12 lanes to the dGPU because 4 have been borrowed (on demand?) by the TB controller? or does the PCH has its PCI lanes? if so, how many? Thanks for the reply.repoman27 - Sunday, July 10, 2011 - link
Anand explained it in his review of the mid 2011 iMac, here: http://www.anandtech.com/show/4340/27inch-apple-im...etamin - Monday, July 11, 2011 - link
I see...I never noticed the PCH/SB always had PCI lanes of its ownrepoman27 - Saturday, July 9, 2011 - link
"At the end of a Thunderbolt chain you can insert a miniDP display, currently the only option is the 27-inch LED Cinema Display but in theory other panels that accept a miniDP input could work as well."Any DisplayPort enabled display will work, and there's plenty of those. You just need to use an asymmetrical cable. Just like you don't need a display with a mini/micro HDMI port to use the mini/micro HDMI out on the devices that have those. Or a PC with mini/micro USB ports.
mAxius - Sunday, July 10, 2011 - link
intel and apple will have thunderbolt the rest of the planet will have external pci express and usb who will winhttp://www.eetimes.com/electronics-news/4217190/PC...
Focher - Sunday, July 10, 2011 - link
According to that article, their standard is due for mid 2013. It's slower than TB and it's not even real. They've just announced plans to make something.repoman27 - Sunday, July 10, 2011 - link
“I'm not entirely convinced that we're limited by Thunderbolt here either - it could very well be the Pegasus' internal controller that's limiting performance.”I’m pretty sure what you’ve gone and done here is bumped into the ceiling imposed by the CPUs in those Macs only supporting a PCIe maximum TLP payload size of 128 bytes. You achieved a little better than 80% of the total 10 Gbps bandwidth available on one Thunderbolt channel in actual data throughput, which is surprisingly good. Even though that bandwidth is exclusive of PCIe’s normal 8b/10b encoding overhead, there’s no getting around the additional overhead inherent to any packetized protocol. A Thunderbolt controller paired with a northbridge that supports 4096 byte payload sizes could theoretically achieve around 99% of the full 10 Gbps.
You’ve also shown that one device using a single Thunderbolt channel can use > 50% of the bandwidth of the 4 PCIe 2.0 lanes connected to the Thunderbolt controller. Thus if you connected one 4-drive SF-2281 Pegasus R6 RAID-0 to each of the Thunderbolt ports on the 2011 iMac, you still shouldn’t expect more than 12,833 Mbps combined throughput.
The Target Disk Mode results are disappointing, although you’re always limited to the speed of the slowest drive that you’re transferring to/from. You didn’t mention what the iMac was packing, but if it’s still just the 1 TB 7200 RPM Seagate that was in the model you reviewed earlier, that would be the limiting factor. Did you check to see what you could pull using FireWire Target Mode between the two?
“simply displaying an image at 60Hz on the 27-inch Cinema Display requires over 6.75Gbps of bandwidth (because of 8b/10b encoding)”
I’m guessing that the 8b/10b encoding overhead is once again not present in the 10 Gbps per channel Thunderbolt bandwidth figure, just as for PCIe packets. Otherwise Thunderbolt would not be able to fully support the DisplayPort 1.1a spec which calls for 10.8 Gbps when including the 8b/10b padding.
“Apple claims that one of the channels is used for DisplayPort while the other is used for PCIe.”
This still flummoxes me. Does that mean that if you daisy chained 2 4-drive SF-2281 Pegasus R6’s to the Thunderbolt port on the MacBook Pro that you would achieve no better than 8021Mbps combined? That neither device could use the bandwidth of the second Thunderbolt channel even with no DisplayPort device present? Also, although Thunderbolt ports only support DisplayPort 1.1a resolutions, might they still support DP 1.2 features such as MST and daisy chainable displays? Or is the only way to connect multiple displays to one Thunderbolt port by using a DP 1.1a multi-display hub and thereby limiting the resolution of at least one of them to less than 2560x1440?
LedHed - Sunday, July 10, 2011 - link
By the time we have a decent amount of devices starting to use the Thunderbolt interface this will be outdated with the 2nd revision. Once again Apple is raising the price for no gain in anyway.LedHed - Sunday, July 10, 2011 - link
Also calling that huge array box mobile is hilarious in itself.xrror - Sunday, July 10, 2011 - link
So Apple will fix all the nagging issues with Thunderbolt connectivity when... they transition to ARM. Begone evil PC people, I'm sure Apple hates it thoroughly that iMacs and MBP can be "perverted" to x86's domain of Windows.So when MacOS basically is superseded by iOS for their "non-handheld mobile devices" and they finally eliminate iMac and MBP since "people who didn't transition to our new taint-ARM/Apple specific processor" line of devices are obviously just lame, as proven by the poor saps holding on to their PowerPC macs. Yea it's coming full circle.
Ugh... I really hope I'm just being paranoid/joking. But...