Evaluation Methodology

​The Lexar Professional Workflow HR2 hub, various card readers, and the memory cards were evaluated using a tweaked version of our traditional direct-attached storage testing workflow. While the hardware (the AnandTech DAS Testbed, whose specifications are outlined in the table below) was reused as-is, we had to make some changes in the software tools used for benchmarking.

AnandTech DAS Testbed Configuration
Motherboard GIGABYTE Z170X-UD5 TH ATX
CPU Intel Core i5-6600K
Memory G.Skill Ripjaws 4 F4-2133C15-8GRR
32 GB ( 4x 8GB)
DDR4-2133 @ 15-15-15-35
OS Drive Samsung SM951 MZVPV256 NVMe 256 GB
SATA Devices Corsair Neutron XT SSD 480 GB
Intel SSD 730 Series 480 GB
Add-on Card None
Chassis Cooler Master HAF XB EVO
PSU Cooler Master V750 750 W
OS Windows 10 Pro x64
Thanks to Cooler Master, GIGABYTE, G.Skill, and Intel for the build components

All the tests were carried out using the Thunderbolt 3 / USB 3.1 Gen 2 Type-C port in the system (enabled by the Alpine Ridge controller). Basic operation was tested out using all combinations of interfaces available in the HR2 hub - USB 3.0 via a Type-C male to Type-A female cable, and, Thunderbolt 2 using a StarTech.com Thunderbolt 3 to Thunderbolt adapter.

The performance evaluation used the readers in the hub via its USB 3.0 interface. However, for the CFast 2.0 reader, the Thunderbolt interface on the reader was directly used and the hub was taken out of the picture.

Our review workflow for memory cards test the following aspects.

  • Sequential writes and reads (fresh pass)
  • robocopy benchmarks (AnandTech DAS Suite) with performance consistency evaluation
  • PCMark 8 Storage Bench (selected workloads)
  • Sequential writes and reads (used pass)
  • CrystalDiskMark sequential workloads (used and refreshed)

The justification for each of the tests and the details of how they are framed is discussed below.

The evaluation of memory cards need to go beyond the usual robocopy benchmarks with data of varying types because the primary use-case is very different. The workload in content capture devices is primarily sequential with a large block size. While the recording involves writes, the import process relies on sequential reads. Storage media usually perform way better (in terms of bandwidth) in sequential accesses compared to random small-sized accesses. Flash-based media is no different, but the challenge in sequential workloads is to maintain performance consistency over the full capacity of the drive when the accesses are sustained. In SSDs and USB thumb drives, we often see performance go down drastically once the SLC cache is filled up. This type of performance is dangerous for content capture devices in the field, as the last thing that photographers and videographers want is the memory card not being able to keep up with the data rate at which the recorded content is written. Since it is a real-time operation, hiccups on the card side tend to result in lost footage. Our first test involves a sequential write FIO [ http://bluestop.org/fio/ ] workload with a block size of 128K and one file open at a time. The workload size is set to 90% of the card capacity, with new files created every 2GB. The sequential write workload is followed by a sequential read of the same set of files with similar access parameters. Throughout this FIO test, the instantaneous write and read bandwidths to the card are recorded and graphed. This gives us a quick overview of the sustained bandwidth numbers for the card - in particular, readers need to note the valleys and ensure that the bitrate at which their camera records on to the card is never more than the minimum achieved write bandwidth.

The above test is followed up with our standard direct-attached storage benchmark. The first part involves transfer of each of the three folders below from a RAM drive in the testbed into the card and reading the data back. This process is repeated thrice for each folder, and the instantaneous bandwidth is graphed.

  • Photos: 15.6 GB collection of 4320 photos (RAW as well as JPEGs) in 61 sub-folders
  • Videos: 16.1 GB collection of 244 videos (MP4 as well as MOVs) in 6 sub-folders
  • BR: 10.7 GB Blu-ray folder structure of the IDT Benchmark Blu-ray (the same that we use in our robocopy tests for NAS systems)

The intent behind this test is to determine if higher queue depths that still correspond to valid real-world use-cases can extract more performance out of the card compared to our primary use-case test. It also serves as a starting point for simulating extended usage of the card.

The second part of our standard DAS benchmark is Futuremark's PCMark 8 Storage Bench. The storage workload involves games as well as multimedia editing applications - this goes well with the microSD use-cases in mobile devices and embedded systems (and the reason for the introduction of the application performance class ratings). The command line version allows us to cherry-pick storage traces to run on a target drive. We chose the following traces.

  • Adobe Photoshop (Light)
  • Adobe Photoshop (Heavy)
  • Adobe After Effects
  • Adobe Illustrator

Usually, PC Mark 8 reports time to complete the trace, but the detailed log report has the read and write bandwidth figures which we present in our performance graphs. Note that the bandwidth number reported in the results don't involve idle time compression. In addition, the storage bench also preconditions the storage device. The whole process provides some interesting bandwidth numbers and also simulates extended usage of the card.

The FIO benchmarks are again repeated at this stage to get an idea of the performance degradation after extended usage. Finally, we run CrystalDiskMark's default sequential workloads in the same state to gather some numbers instead of inferring them from the graph alone. We try to then restore the card to its original performance state (method varies from one card type to another) and check if there is any improvement in the CrystalDiskMark numbers.

In addition to a qualitative and quantitative discussion of the performance numbers, pricing is also an important aspect. Each card being reviewed is compared on a $/GB metric to provide readers with a measure of the value proposition.

Introduction Lexar Professional 1066x CompactFlash Performance
POST A COMMENT

32 Comments

View All Comments

  • petteyg359 - Wednesday, June 28, 2017 - link

    Or you could get four $10 card reader / hub combos that nut only are just as fast, but also daisy chain together in less space than this monstrosity. Reply
  • Zak - Thursday, June 22, 2017 - link

    I wonder if the problem, this clumsy solution attempts to address, actually exists. Reply
  • DanNeely - Thursday, June 22, 2017 - link

    Probably not for home or occasional use, but for daily use when time is money it does.

    Saving even a minute or two a day vs a cheap reader adds up fast if you've got a job where you're doing big transfers daily. A dollar a day makes a fully populated TB3 version break even within 2 years at that rate. Once you go beyond a mom and pop size business overhead means hourly rates are probably at least double salary due to overheads; at that point you're looking a breakeven period of months.
    Reply
  • evanrich - Saturday, June 24, 2017 - link

    Funny how people who have no idea what such products are for want to chime in like they have a clue. "I wonder if the problem this ....automobile.... attempts to solve, even exists" - Horse trainer in the 1900's. For people who shoot a lot of photos/video, and need to dump multiple cards in the field quickly, this works well. While I don't have the hub, I own 2 of the SD readers and I use them EVERY time I do a shoot to quickly dump cards and then start shooting again. I can't wait for some shitty usb2.0 reader to dump 256GB of images off to my PC. Reply
  • hieuhef - Monday, June 26, 2017 - link

    Problem: I have 4 memory cards from my shoot that I need to add info to and add extra metadata to as I save it to my PC. I don't want to do it one card at a time.

    Solution: This hub and 4 readers. Add info and metadata to be applied to each photo in Photomechanic, then it saves each image from each card to a HDD on my PC along while backing up to an external drive or RAID in one action. That is a lot of time saved over a course of a year.
    Reply
  • petteyg359 - Wednesday, June 28, 2017 - link

    Cheaper and smaller solution: daisy chain individual card reader / hubs, or use multiple single card readers in a single hub. There are plenty of card readers out there that are at least as fast as this monstrosity. Reply
  • RaichuPls - Thursday, June 22, 2017 - link

    So uhm. What's with the lack of articles recently? A10 deep dive, GS8 review, U11, GTX 1050/1050Ti, hell even a 2016 Macbook Pro review? iMac Pro? OnePlus 5? Reply
  • fanofanand - Thursday, June 22, 2017 - link

    They had a decently in-depth review of the new Intel chips, but I too have noticed a lack of recent content the last couple of weeks. Reply
  • rtho782 - Friday, June 23, 2017 - link

    I'm still waiting for the GTX960 review. Any day now. Reply
  • evanrich - Saturday, June 24, 2017 - link

    Don't exactly see you contributing any articles there yourself there bro... Reply

Log in

Don't have an account? Sign up now