Evaluating Futuremark's Servermark VDI on the Supermicro SYS-5028D-TN4T
by Ganesh T S on September 1, 2016 8:00 AM EST- Posted in
- Benchmarks
- Futuremark
- Supermicro
- Servers
- Xeon-D
Benchmarking the Supermicro SYS-5028D-TN4T
The guides and scripts that are supplied with Servermark VDI assume a VMWare-based virtualization setup. However, for our Hyper-V setup, the scripts had to be reworked in Powershell. Two distinct scripts were developed. The first script took a base VM and cloned it N times with different copies of the base image (not differencing VHDs). This script was run on the Supermiro SYS-5028D-TN4T.
An Intel D54250WYK Haswell NUC running Windows 8.1 Pro x64 was used as the console node on the main network. The second script was run on this machine. It started the VMs in sets of 2 (start with 2, then 4, 6 and so on till a configurable number of VMs based on N in the first script) and processed the ServermarkControlConsole command with the appropriate arguments based on the number of active powered on VMs in that particular iteration. One of the challenges here was determining the DHCP IPs of the active VMs on their main network interface to prepare the arguments.
The Xeon-D 1540 is a 8C/16T processor, and our system was configured with 64GB of RAM. To keep things reasonable, we set the number of maximum simultaneously active VMs to be 16. Each of the Hyper-V Gen 2 VMs was allocated one vCPU and 2GB of RAM. Other VM configurations were left at default. The graph below plots the average PCMark score obtained on each VM as a function of the number of simultaneously active VMs.
Other statistics from the benchmark runs are summarized in the table below.
| Supermicro SYS-5028D-TN4T - Futuremark Servermark VDI 0.4.3 - PCMark Scores | |||||
| No. of Active VMs | Minimum | Maximum | Average | Median | Standard Deviation (%) |
| 2 | 1701.0 | 1777.0 | 1742.0 | 1749.0 | 1.73 |
| 4 | 1643.0 | 1753.0 | 1699.0 | 1698.0 | 2.01 |
| 6 | 1637.0 | 1733.0 | 1686.0 | 1687.0 | 1.63 |
| 8 | 1595.0 | 1733.0 | 1669.0 | 1674.0 | 2.03 |
| 10 | 1544.0 | 1694.0 | 1642.0 | 1640.0 | 2.21 |
| 12 | 1568.0 | 1702.0 | 1628.0 | 1625.0 | 1.84 |
| 14 | 1562.0 | 1687.0 | 1621.0 | 1622.0 | 1.71 |
| 16 | 1532.0 | 1692.0 | 1605.0 | 1606.0 | 1.83 |
A number of tests were carried out by allocating more vCPUs per VM and also allowing dynamic memory amounts for each according to their requirements. Irrespective of the VM configuration, we were unable to score more than the 2000 required for Servermark VDI to certify the server. This is likely due to the single-threaded performance of the Xeon-D 1540 not being good enough for the generic office workloads from PCMark 8's 'Work' preset.
Facebook
Twitter
RSS
18 Comments
View All Comments
Ryan Smith - Thursday, September 1, 2016 - link
Ultimately this is an article we worked on because Futuremark's software looked neat, and we wanted to see if it could tell us anything useful.We're nerds at heart. We like looking at new things.=)
aaronb1138 - Friday, September 2, 2016 - link
Although FM puts "2000" as their recommended benchmark level, I think you will find for VDI that the bar is much, much lower. The main gist of VDI is the fact that most users need very little computational power continuously. They need the boot and load software bumps and then most productivity software sips at a few percent of CPU. I've noticed web browsers, especially Chrome, have become the biggest consumers of CPU on workstations, and it seems almost vindictive the way certain browsers will hose a system when resource starved. Mozilla/Firefox is especially designed to make virtualization and RDC scenarios bad (the thousands of cache folders is intentionally shit design for roaming profiles and causes huge login hangs).I noticed a NUC gets around 5000 in the benchmark. From my experience in VDI as well as XenApp and RDS architectures, a NUC could support around 10 users with sufficient RAM (16GB would do it - 32GB and we're talking 20+ users, maybe 40 in XenApp/RDS instead of VDI).
Also, for the love of all that is holy, do not benchmark against RAID 0, it invalidates much of the testing. Test against both RAID 10 and 5 as those are industry standards for such infrastructures.
If you really want to deep dive, check out differences in memory consumption in HyperV vs VMware with VDI. While both support memory deduplication, experience shows that HyperV is much faster and more effective at VDI memory dedupe, while VMware seems to rely a tad heavy on memory ballooning. In server farm / VDI situations, RAM and storage I/O are ALWAYS more limiting than CPU throughput. You're not putting Pro-E and Solidworks into a VDI farm most of the time, just MS Office, a line of business application, and a browser.
madisson - Thursday, September 1, 2016 - link
maybe i am approaching this too focused on a consumer's mind, but all of that hardware needs to be shipped, unpacked, installed, and plugged in. For me, I prefer to log-in to a cloud PC for gaming and virtual PC. There are several services similar to what I'm talking about, www.paperspace.com being the one I happened to pick.I'm sure FM goes for more enterprise-level agreements, but what about the small teams and individuals?
name99 - Thursday, September 1, 2016 - link
It seems to me this is completely a WINTEL benchmark.That's fine for a certain class of users, I guess, but is, I think, a lot less interesting for many readers. It does nothing to clarify whether either
- a Linux PC would be a better server solution OR
- whether an ARM server could handle a particular load.
Obviously AnandTech can do what they want in their reviews, but I, for one, am a lot less interested in comparisons that are focussed on the few percent differences between Wintel box A and Wintel box B than in comparisons between very different systems.
powerarmour - Thursday, September 1, 2016 - link
Exactly.aaronb1138 - Friday, September 2, 2016 - link
Well, they were looking at VDI. I don't know what kind of monster would try VDI+Linux+end user workloads in the workplace. But yeah, keep saying dumbass shit about Linux and ARM.gabemcg - Tuesday, September 6, 2016 - link
Lots of hostility unfairly being directed at the authors/editors here. I thought using the benchmark preview as a way to couch a hardware review was genuinely interesting both as a nerd with a home lab, and a professional wanting to keep abreast of how the enterprise class gear is being evaluated. Keep up the good work, haters gonna hate!simran sidhki - Thursday, October 18, 2018 - link
hello