Java Server Performance

The SPECjbb 2013 benchmark has "a usage model based on a world-wide supermarket company with an IT infrastructure that handles a mix of point-of-sale requests, online purchases, and data-mining operations." It uses the latest Java 7 features and makes use of XML, compressed communication, and messaging with security.

Benchmark architecture diagram

We tested with four groups of transaction injectors and back-ends. We applied relatively basic tuning to mimic real-world use. We used this JVM configuration setting for the systems limited to 32 GB (all Xeon E3):

"-server -Xmx4G -Xms4G -Xmn2G -XX:+AlwaysPreTouch -XX:+UseLargePages"

With these settings, the benchmark takes about 20-27GB of RAM. For the servers that could address 64 GB or more (Atom, Xeon D and Xeon E5), we used a slightly beefier setting:

"-server -Xmx8G -Xms8G -Xmn4G -XX:+AlwaysPreTouch -XX:+UseLargePages"

With these settings, the benchmark takes about 43-57GB of RAM. The first metric is basically maximum throughput.

SPECJBB 2013-Multi max-jOPS

As long as you run enough JVMs on top your server, the Xeon D and Xeon E5 will not dissapoint. The Xeon D is at least 37% faster than the previous Xeon E3 generation, the Xeon E5 delivers 50% more. 

The Critical-jOPS metric is a throughput metric under response time constraint.

SPECJBB 2013-Multi Critical-jOPS

The Xeon D seems to be slightly hindered by the lack of memory bandwidth in the max throughput benchmark, but less than in our HPC benchmark. It is important to understand that maximum throughput is very important in a HPC benchmark, but for a Java based back-end server, the critical benchmark matters much more than the maximum one. The reason is simple: the critical benchmark tells you what your customers will experience on a daily basis, the maximum throughput benchmark descibes what you will get in the worst case scenario when your server is pushed to its limits. 

In the critical benchmark, the Xeon D is at least 65% faster than any Xeon E3. The Broadwell core is a minor improvement over the Haswell core when you look at performance only (single threaded integer performance), but once it is integrated in a chip like the Xeon D, it is astonishing how much performance per watt you get. A 60-70% increase in performance per watt is a rare thing indeed. 

SPECJBB®2013 is a registered trademark of the Standard Performance Evaluation Corporation (SPEC).
HPC: Fluid Dynamics Web Server Performance
Comments Locked

90 Comments

View All Comments

  • Flunk - Tuesday, June 23, 2015 - link

    Yes, but it's still bad marketing. -D is associated with inferior, overly hot, bad performing Intel chips.
  • IanHagen - Tuesday, June 23, 2015 - link

    Certainly. From a marketing standpoint it's a pretty poor choice. I agree with wussupi, E4 would haven been a far better name.
  • karpodiem - Tuesday, June 23, 2015 - link

    does anyone know where to buy these online? I'm looking for just the board/processor, model # 'X10SDV-TLN4F'

    All these random/small Supermicro resellers are selling it now, based on some Google searches. They're marking it up in price by at least a hundred bucks, because availability is limited. Anyone know when Newegg might get it in stock?

    Looking to do a FreeNAS build - this board + IBM M1015 card in an ATX motherboard (6x4TB drives in RAIDZ2).
  • ats - Tuesday, June 23, 2015 - link

    The TLN4F is the one in most demand and almost no place is able to keep it in stock. There are multiple places that will order it for you for ~1K but wait times can be anywhere from 1 week to 1 month.
  • Jon Tseng - Tuesday, June 23, 2015 - link

    > And the reality is that the current SoCs with an ARM ISA do not deliver the necessary per core
    > performance: they are still micro server SoCs, at best competing with the Atom C2750. So
    > currently, there is no ARM SoC competition in the scale out market until something better than
    > the A57 hits the market for these big players.

    Dude... You really want to have a look at the latest ThunderX parts or the X-Gene 16nm shrinks before you start making unwise statements like that. These aren't waiting around for A57 they are custom ARM architecture designs. Per core performance might not be as hot as Xeon but once you start to throw 48 cores on a die I wouldn't quite call that "at best competing with Avaton".
  • smoohta - Tuesday, June 23, 2015 - link

    Link to reviews?
  • ats - Tuesday, June 23, 2015 - link

    X-Gene is in the article, any further shrinks are still entirely vapor. ThunderX isn't currently available is is likely to have significantly worse per core performance than Atom C2k series and worse than A57. All the cores in the world don't do jack if the ST isn't there. And ST performance IS a barrier even in scale out. For general scale out, C2750 was found fairly wanting because of the ST performance, and neither X-Gene nor ThunderX even compete with C2750 in ST performance... QED.
  • mczak - Tuesday, June 23, 2015 - link

    He said "currently". The X-Gene 16nm cores might offer some competition who knows - but those are X-Gene 3 whereas you can't even buy anything with X-Gene 2 28nm ones right now... Likewise, ThunderX servers have been announced, but I haven't seen any reviews yet.
  • name99 - Tuesday, June 23, 2015 - link

    Look at the ThunderX parts HOW? Cavium releases fsck-all information about them. No-one knows if they are even OoO, how wide they are, etc.
    Yes, there are 48 cores on a SoC; and presumably they will do well for tasks like memcached that like lots of low-performance parallelism. But right now, we have ZERO evidence that a ThunderX part is a better single-threaded core than A57, let alone that it's comparable to Broadwell.
  • der - Tuesday, June 23, 2015 - link

    NOICE FAM!

Log in

Don't have an account? Sign up now