Scale-Out Big Data Benchmark: ElasticSearch

ElasticSearch is an open source, full text search engine that can be run on a cluster relatively easy. It's basically like an open source version of Google Search that can be deployed in an enterprise. It should be one of the poster children of scale-out software and is one of the representatives of the so called "Big Data" technologies. Thanks to Kirth Lammens, one of the talented researchers at my lab, we have developed a benchmark that searches through all the Wikipedia content (+/- 40GB). Elasticsearch is – like many Big Data technologies – built on Java (we use the 64-bit server version 1.7.0).

Elastic Search

The term "Big data" almost immediately suggests that you need massive machines, more like the new Xeon E7 which supports up to 6 TB. In reality, many big data analyses are running on top of very humble machines in a cluster. ElasticSearch is such an an application: the underlying Java technology does not work well with a larger than 32 GB heap. A total of 64 GB RAM is considered as the sweet spot, to leave some RAM space for filesystem caching. 

The result of the Xeon D is stunning. The Xeon D is no less than 70% faster than the fastest Xeon E3s. Better performance is possible with the Xeon E5, but the price tag of those servers is not comparable to the Xeon D servers. The Xeon D-1540 (and as a result the SYS-5028D-TN4T) is the performance per dollar champ here. 

Web Server Performance Idle Power
Comments Locked

90 Comments

View All Comments

  • julianb - Saturday, October 31, 2015 - link

    Thanks for the reply, man.
    And sorry for my late reply, totally forgot about this thread :)
  • eva2000 - Tuesday, June 23, 2015 - link

    Nice... Xeon D-1540 is awesome, but I wish it was clocked 0.2Ghz higher across the board would be just enough to tip that scale versus E5. Did my own benchmarks at https://community.centminmod.com/threads/2864/ :)
  • extide - Wednesday, June 24, 2015 - link

    Thats probably exactly why it ISNT clocked 0.2Ghz higher across the board ;)

    I'm sure Intel wants to see some space between this and E5.
  • boogerlad - Tuesday, June 23, 2015 - link

    If this was marketed for the consumer market with the ability to overclock, this would outsell everything completely. This is what the enthusiast needs!!!
  • Refuge - Tuesday, June 23, 2015 - link

    I don't think this is going to do much of anything for an enthusiast.

    Unless they are interested in building a server for some experiment or project.
  • JohanAnandtech - Wednesday, June 24, 2015 - link

    I still think the i7 59xx series is a better match for consumers: higher clocks and thus ST performance. The Xeon D most interesting features such as integrated 10 GBe and low power don't interest most performance consumers. Most people will have a hard time saturating a 1 GBe line and power savings are not a priority.
  • tspacie - Wednesday, June 24, 2015 - link

    Seems to tick all the boxes for a software development machine. Very good at compilation. Reasonably priced for the performance. Low power. ECC memory. I'm tempted
  • extide - Wednesday, June 24, 2015 - link

    EXACTLY what I was thinking!
  • MrSpadge - Saturday, June 27, 2015 - link

    I would be very tempted by such a chip as well, using it for BOINC. However, Broadwell looses some of the power efficiency advantage if you push it harder, i.e. the largest gains are at low and moderate frequency. Perfect for such server chips and mobile ones, but not so much for people aiming for 4+ GHz.
  • MaxKreimerman - Tuesday, June 23, 2015 - link

    Sounds impresive in just 45w package, but imposible to find in the retail sites such as newegg or wiredzone

Log in

Don't have an account? Sign up now