Java Performance

The SPECjbb 2015 benchmark has 'a usage model based on a world-wide supermarket company with an IT infrastructure that handles a mix of point-of-sale requests, online purchases, and data-mining operations'. It uses the latest Java 7 features and makes use of XML, compressed communication, and messaging with security.

We test SPECjbb with four groups of transaction injectors and backends. The reason why we use the "Multi JVM" test is that it is more realistic: multiple VMs on a server is a very common practice.  

The Java version was OpenJDK 1.8.0_222. We used the older JDK 8 as the most recent JDK 11 has removed some deprecated JAVA EE modules that SPECJBB 1.01 needs.  We applied relatively basic tuning to mimic real-world use, while aiming to fit everything inside a server with 128 GB of RAM:

"-server -Xmx24G -Xms24G -Xmn16G -XX:+AlwaysPreTouch -XX:+BiasedLocking"

We tested with huge pages on and off. 

The graph below shows the maximum throughput numbers for our MultiJVM SPECJbb test. Since the test is almost identical to the one that we have used in our ThunderX2 review (JDK8 1.8.0_166), we also include Cavium's server CPU. 

Ultimately we publish these numbers with a caveat: you should not compare this with the official published SPECJBB2015 numbers, because we run our test slightly differently to the official run specifications. We believe our numbers make as much sense (and maybe more) as most professionals users will not go for the last drop of performance. Using these ultra optimized settings can result in unrepeateable and hard to debug inconsistent errors - at best they will result in subpar performance as they are so very specific to SPECJBB. It is simply not worth it, a professional will stick with basic and reliable optimization in the real non-HPC world. In the HPC world, you simply rerun your job in case of an error. But in the rest of the enterprise world you just made a lot users very unhappy and created a lot of work for (hopefully) well paid employees. 

SPECJBB 2015-Multi Max-jOPS Huge Pages Impact

The EPYC 7742 performance is excellent, outperforming the best available Intel Xeon by 48%.

Notice that the EPYC CPU performs better with small pages (4 KB) than with large ones (2 MB). AMD's small pages TLB are massive and as result page table walks (PTW) are seldom with large pages. If the number of PTW is already very low, you can not get much benefit from increasing the page size. 

What about Cavium? Well, the 32-core ThunderX2 was baked with a 16 nm process technology. So do not discount them - Cavium has a unique opportunity as they move the the ThunderX3 to 7 nm FFN TSMC too. 

To be fair to AMD, we can improve performance even higher by using numactl and binding the JVM to certain CPUs. However, you rarely want to that, and happily trade that extra performance for the flexibility of being able to start new JVMs when you need them and let the server deal with it. That is why you buy those servers with massive core counts. We are in the world of micro services, docker containers, not in the early years of 21st century. 

Ok, what if you do that anyway? AMD offered some numbers, while comparing them to the officialy published SPEJBB numbers of Lenovo ThinkSystem SR650 (Dual Intel 8280). 

AMD achieves 335600 by using 4 JVM per node, binding them to "virtual NUMA nodes". 

Just like Intel, AMD uses the Oracle JDK, but there is more to these record breaking numbers. A few tricks that only benchmarking people can use to boost SPECJBB: 

  • Disabling p-states and setting the OS to maximum performance (instead of balanced)
  • Disabling memory protection (patrol scrub)
  • Using older garbage collector because they happen to better at Specjbb
  • Non-default kernel settings 
  • Aggressive java optimizations 
  • Disabling JVM statistics and monitoring
  • ...

In summary, we don't think that it is wise to mimic these settings, but let us say that AMD's new EPYC 7742 is anywhere between 48 and 72% faster. And in both cases, that is significant!

Legacy: 7-zip Java Performance: Critical-jOPS


View All Comments

  • negusp - Wednesday, August 7, 2019 - link

    hard F in the chat for intel Reply
  • pancakes - Wednesday, August 7, 2019 - link

    F in chat for wallets of people running Windows server Reply
  • azfacea - Wednesday, August 7, 2019 - link

    windows server in 2019 LUL Reply
  • diehardmacfan - Wednesday, August 7, 2019 - link

    on-prem Windows Server is probably at an all time high in 2019? Reply
  • azfacea - Thursday, August 8, 2019 - link

    desperate for a comeback huh? cool hold your 10% tight and gloat about upcoming bfloat16 Reply
  • diehardmacfan - Thursday, August 8, 2019 - link

    Sorry, who is desperate for a comeback? Bring up a floating point format when called out on the ridiculous notion that Windows Server isn't still a large part of the marketplace? say wha Reply
  • mkaibear - Thursday, August 8, 2019 - link

    Just hopping in to say that I am an IT manager for a major employer in the UK and of our 1800 servers more than 80% of them are Windows... this is not a trend which I see changing any time soon. Reply
  • Deshi! - Thursday, August 8, 2019 - link

    I work as an application engineer for a major global finance company that develops and hosts banking and e-commerce software used by banks and major shopping outlets. 90% of all our servers are either Linux or AIX mainly running websphere or standalone Java instances. We only have a handful of Windows servers, mainly for stuff like active directory and Outlook/ SharePoint. So yeah allot of it depends on the use case, but allot of the big boys do use Linux or AIX. It's cheaper and performs better for these use cases. Reply
  • cyberguyz - Thursday, August 8, 2019 - link

    I guess we all have to ask ourselves, who are the customers that would benefit most from a 64-core, 128 gen 4 PCIe processors? SMB or huge customers that would shell out many millions of $$$ for their middleware & backend systems? @Deshi! I or one of my L3 colleagues an L3 engineer contacted by your global finance company to fix Websphere problems some years back ;) Reply
  • FreckledTrout - Thursday, August 8, 2019 - link

    @cyberguz, Who would benefit from these high core servers? Any company running VM's so pretty much every large company. This goes doubly for cloud providers. Reply

Log in

Don't have an account? Sign up now