Server Buying Decisions: Memory
by Johan De Gelas on December 19, 2013 10:00 AM EST- Posted in
- Enterprise
- Memory
- IT Computing
- Cloud Computing
- server
CDN
Serving static files is a common task for web servers. However, it is best to offload this from the application web servers to dedicated web servers that can be configured optimally. Those servers, which together form the Content Delivery Network (CDN), are placed in networks closer to the end-user, saving IP transport costs and improving response times. These CDN servers need to cache files as much as they can, and are thus ideal candidates for some high capacity DIMM testing.
A big thanks to Wannes De Smet, my colleague at the Sizing Servers Lab, part of Howest University for making the CDN test a reality.
Details of the CDN setup
We simulate our CDN test with one CDN server and three client machines in a 10 Gbit/s network. Each client machine simulates thousands of users requesting different files. Our server runs Ubuntu 13.04 with Apache. Apache was chosen over NginX as it uses the file system cache whereas NginX does not. Apache has been optimized to provide maximum throughput.
Apache is set to use the event multi process model, configured as follows:
StartServers 1
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 150
ThreadsPerChild 150
MaxClients 1500
MaxRequestsPerChild 0
Each client machine has an X540-T1 10Gbit adapter connected to the same switch as the HP DL380 (see benchmark configuration). To load balance traffic across the two links, the balanced-rr algorithm is selected for the bond. No network optimization has been carried out, as network conditions are optimal.
The static files requested originate from sourceforge.org, a mirror of the /a directory, containing 1.4TB of data or 173000 files. To model real-world load for the CDN two different usage patterns or workloads are executed simultaneously.
The first workload contains a predefined list of 3000 files from the CDN with a combined concurrency of 50 (50 requests per second), resembling files that are “hot” or requested frequently. These files have a high chance to reside in the cache. Secondly, another workload requests 500 random files from the CDN per concurrency (maximum concurrent requests is 45). This workload simulates users that are requesting older or less frequently accessed files.
Test Run
An initial test run will deliver all files straight off an iSCSI LUN. The LUN consists of a Linux bcache volume, with an Intel 720 SSD as cache and three 1TB magnetic disks in RAID-5, handled by an Adaptec ASR72405 RAID adapter, as backing device. Access is provided over a 10Gbit SAN network. Handling these files will fill the page cache, thus filling memory. Linux will automatically use up to 95% of the available RAM memory.
Subsequent runs are measured (both throughput and response time) and will deliver as many files from memory as possible. This depends on the amount of RAM installed and how frequent a file is requested, as the page cache uses a Least Recently Used algorithm to control which pages ought to remain in cache.
Facebook
Twitter
RSS
27 Comments
View All Comments
subflava - Thursday, December 19, 2013 - link
Great article...look forward to more enterprise/IT professional based articles from Anandtech in the future. This is very timely for me as my company is just about to pull the trigger on a server upgrade. Interesting stuff.JohanAnandtech - Friday, December 20, 2013 - link
Thanks for sharing! :-)DERSS - Friday, December 27, 2013 - link
You guys are seriously super-cool; thanks.wsaenotsock - Thursday, December 19, 2013 - link
costed?blaktron - Thursday, December 19, 2013 - link
Good article, although as an enterprise architect, I can tell you the one true benefit to LRDIMMS is in 2 and 4 socket vhost builds, because the double density RAM gives you the freedom to turn off NUMA spanning and still get near-ideal guest density.Almost nobody runs caching servers that big, although at almost double performance over a 256GB build (the 100k + concurrent user norm) its kind of attractive to run 2 of these per DC instead of 6 smaller ones (which would actually be the real world comparison with those kind of deltas).
mexell - Thursday, December 19, 2013 - link
Real-world pricing, at least in the enterprise context, is quite a bit off from your numbers. In my employer's price bracket, we regularily buy similar servers as your 24*16GB config for about the same price (13k€) - but including a 3 year subscription VMWare Enterprise license, which is about 6 to 7 k€ on its own. No one pays list price on that kind of hardware.JohanAnandtech - Friday, December 20, 2013 - link
Are you sure that there is not a big discount on the VMware license? And smaller enterprises will pay something close to the list price. I know that the typical discount is 10-20% for smaller quantities, not more.blaktron - Friday, December 20, 2013 - link
Depends on the country Johan. The partner channel managers get to decide discounts on partner orders (which he is describing). Also, the bundling discount doesn't happen everywhere, but I could buy that server for like $15k CDN.The VMware license cost seems out of this world to me too, because we license our hosts for anyone from 2500 to 5k CDN, depending on their agreement with VMware.
mexell - Saturday, December 21, 2013 - link
I don't really know where exactly the discount is applied, as the licenses are OEM and we don't get line-item pricing. In our market segment (large enterprise with Dell, medium-to-large with HP) we usually see at least 40% off on list prices, in some cases (networking equipment) up to 75%.VMWare, on the other hand, is especially rigid with their pricing structure. Two years ago, when we negotiated for a 100 host branch office deployment, they referred to their list pricing. For them, we are not even big enough to speak directly to us.
dstarr3 - Thursday, December 19, 2013 - link
Wow. With 768GB of memory, I bet you could run Crysis.