Scale-Out Big Data Benchmark: ElasticSearch

ElasticSearch is an open source, full text search engine that can be run on a cluster relatively easy. It's basically like an open source version of Google Search that can be deployed in an enterprise. It should be one of the poster children of scale-out software and is one of the representatives of the so called "Big Data" technologies. Thanks to Kirth Lammens, one of the talented researchers at my lab, we have developed a benchmark that searches through all the Wikipedia content (+/- 40GB). Elasticsearch is – like many Big Data technologies – built on Java (we use the 64-bit server version 1.7.0).

Elastic Search

The term "Big data" almost immediately suggests that you need massive machines, more like the new Xeon E7 which supports up to 6 TB. In reality, many big data analyses are running on top of very humble machines in a cluster. ElasticSearch is such an an application: the underlying Java technology does not work well with a larger than 32 GB heap. A total of 64 GB RAM is considered as the sweet spot, to leave some RAM space for filesystem caching. 

The result of the Xeon D is stunning. The Xeon D is no less than 70% faster than the fastest Xeon E3s. Better performance is possible with the Xeon E5, but the price tag of those servers is not comparable to the Xeon D servers. The Xeon D-1540 (and as a result the SYS-5028D-TN4T) is the performance per dollar champ here. 

Web Server Performance Idle Power
Comments Locked

90 Comments

View All Comments

  • Kjella - Tuesday, June 23, 2015 - link

    Server on a chip? It's not intended for use with a display, it does all it's "supposed to" do for the hyperscale market without any display.
  • close - Tuesday, June 23, 2015 - link

    "Intel was able to combine 8 of them together with dual 10 Gbit, 4 USB 3.0 controllers, 6 SATA 3 controller and quite a bit more".
    This ^^ makes it a SoC. Ok, a video output would be nice but that certainly doesn't disqualify it.
  • ats - Tuesday, June 23, 2015 - link

    cause video isn't required or even wanted in this market segment. It is a SoC, which simply means system on a chip and doesn't have some ironclad definition. Hell, most "SoC" chips aren't really systems on a chip anyways and require significant supporting logic (this is true for just about any cell phone SoC on the market too).
  • bill.rookard - Tuesday, June 23, 2015 - link

    Exactly, you would tend to use remote management over the network to admin this type of a unit. I have several rackmounted servers in my basement (I do some home-serving of websites over a business class connection) and while I do have them actually hooked up to a display, I can hardly remember the last time I looked at them as 99.9% of the time I SSH into everything for administration.

    About the only time you'd ever really use a display is if you were doing multiple VMs of assorted types. Beyond that, it's wattage wasted.
  • ats - Tuesday, June 23, 2015 - link

    Yeah honestly, having several SM boards with their ILM system, the only time I'd ever hook up a display is if the network was down. The SM ILM will fully proxy pretty much anything you want and give you a 1200p display that works for just about anything. And you can remotely hook up CDs, DVDs, BRs, USB, etc through it along with the stand console and keyboard/mouse functions. Its a very nice solution.
  • nightbringer57 - Tuesday, June 23, 2015 - link

    Basically, you don't need video output.
    Even if you do, mainboard manufacturers usually include a third-party chip with dedicated functions that, along other things, provide a VGA port usable for a server use.
    In this case, the AST2400 chip offers some basic GPU functions with a VGA port along with many remote control-related stuff.
    Adding all those functions to the Intel SoC would be awfully expensive. The chip only requires a simple PCIe x1 connection from the SoC, but provides hundreds of additional pins. Not only would those functions probably be hard to implement on a relatively recent 14nm process, but it would require at least 300 new pins on the SoC to add all the 3rd party chip's functions on it, which is almost impossible to do.
  • Th-z - Tuesday, June 23, 2015 - link

    There doesn't seem to have a concrete definition for the term SoC, but it's ridiculous now with the term SoC bandwagon. Everything seems to be called "SoC" these days as long as a chip has more than one functions integrated. One of examples is people even called current console's integrated CPU and GPU chip as SoC, which doesn't even have networking and other peripheral units in it. When a system has so many "SoCs" inside, the term really has lost its meaning and significance.
  • redzo - Tuesday, June 23, 2015 - link

    I'm thinking this is a bad name for a product like this. It reminds of the infamous Celeron D and Pentium D line.
  • nandnandnand - Tuesday, June 23, 2015 - link

    Anyone who can figure out Xeon D exists can probably tell the difference
  • wussupi83 - Tuesday, June 23, 2015 - link

    I agree with redzo, I think anyone who can figure out a 'Xeon D' exists AND remembers that Pentium & Celeron D's existed would initially assume this is a budget Xeon - which it's clearly not. E4 sounds pretty logical. But sure lets just put D...

Log in

Don't have an account? Sign up now