Office and Science

In this version of our test suite, all the science focused tests that aren’t ‘simulation’ work are now in our science section. This includes Brownian Motion, calculating digits of Pi, molecular dynamics, and for the first time, we’re trialing an artificial intelligence benchmark, both inference and training, that works under Windows using python and TensorFlow.  Where possible these benchmarks have been optimized with the latest in vector instructions, except for the AI test – we were told that while it uses Intel’s Math Kernel Libraries, they’re optimized more for Linux than for Windows, and so it gives an interesting result when unoptimized software is used.

Agisoft Photoscan 1.3.3: link

The concept of Photoscan is about translating many 2D images into a 3D model - so the more detailed the images, and the more you have, the better the final 3D model in both spatial accuracy and texturing accuracy. The algorithm has four stages, with some parts of the stages being single-threaded and others multi-threaded, along with some cache/memory dependency in there as well. For some of the more variable threaded workload, features such as Speed Shift and XFR will be able to take advantage of CPU stalls or downtime, giving sizeable speedups on newer microarchitectures.

For the update to version 1.3.3, the Agisoft software now supports command line operation. Agisoft provided us with a set of new images for this version of the test, and a python script to run it. We’ve modified the script slightly by changing some quality settings for the sake of the benchmark suite length, as well as adjusting how the final timing data is recorded. The python script dumps the results file in the format of our choosing. For our test we obtain the time for each stage of the benchmark, as well as the overall time.

(1-1) Agisoft Photoscan 1.3, Complex Test

GeekBench 5: Link

As a common tool for cross-platform testing between mobile, PC, and Mac, GeekBench is an ultimate exercise in synthetic testing across a range of algorithms looking for peak throughput. Tests include encryption, compression, fast Fourier transform, memory operations, n-body physics, matrix operations, histogram manipulation, and HTML parsing.

I’m including this test due to popular demand, although the results do come across as overly synthetic, and a lot of users often put a lot of weight behind the test due to the fact that it is compiled across different platforms (although with different compilers).

We have both GB5 and GB4 results in our benchmark database. GB5 was introduced to our test suite after already having tested ~25 CPUs, and so the results are a little sporadic by comparison. These spots will be filled in when we retest any of the CPUs.

(8-1c) Geekbench 5 Single Thread(8-1d) Geekbench 5 Multi-Thread

We saw a few instances where the 35W/45W results were almost identical, with the margin that the 35W would come out ahead in single threaded tasks. This may be because 35W was a fixed setting in the software options, whereas 45W was the power management framework in action.

NAMD 2.13 (ApoA1): Molecular Dynamics

One of the popular science fields is modeling the dynamics of proteins. By looking at how the energy of active sites within a large protein structure over time, scientists behind the research can calculate required activation energies for potential interactions. This becomes very important in drug discovery. Molecular dynamics also plays a large role in protein folding, and in understanding what happens when proteins misfold, and what can be done to prevent it. Two of the most popular molecular dynamics packages in use today are NAMD and GROMACS.

NAMD, or Nanoscale Molecular Dynamics, has already been used in extensive Coronavirus research on the Frontier supercomputer. Typical simulations using the package are measured in how many nanoseconds per day can be calculated with the given hardware, and the ApoA1 protein (92,224 atoms) has been the standard model for molecular dynamics simulation.

Luckily the compute can home in on a typical ‘nanoseconds-per-day’ rate after only 60 seconds of simulation, however we stretch that out to 10 minutes to take a more sustained value, as by that time most turbo limits should be surpassed. The simulation itself works with 2 femtosecond timesteps. We use version 2.13 as this was the recommended version at the time of integrating this benchmark into our suite. The latest nightly builds we’re aware have started to enable support for AVX-512, however due to consistency in our benchmark suite, we are retaining with 2.13. Other software that we test with has AVX-512 acceleration.

(2-5) NAMD ApoA1 Simulation

AI Benchmark 0.1.2 using TensorFlow: Link

Finding an appropriate artificial intelligence benchmark for Windows has been a holy grail of mine for quite a while. The problem is that AI is such a fast moving, fast paced word that whatever I compute this quarter will no longer be relevant in the next, and one of the key metrics in this benchmarking suite is being able to keep data over a long period of time. We’ve had AI benchmarks on smartphones for a while, given that smartphones are a better target for AI workloads, but it also makes some sense that everything on PC is geared towards Linux as well.

Thankfully however, the good folks over at ETH Zurich in Switzerland have converted their smartphone AI benchmark into something that’s useable in Windows. It uses TensorFlow, and for our benchmark purposes we’ve locked our testing down to TensorFlow 2.10, AI Benchmark 0.1.2, while using Python 3.7.6.

The benchmark runs through 19 different networks including MobileNet-V2, ResNet-V2, VGG-19 Super-Res, NVIDIA-SPADE, PSPNet, DeepLab, Pixel-RNN, and GNMT-Translation. All the tests probe both the inference and the training at various input sizes and batch sizes, except the translation that only does inference. It measures the time taken to do a given amount of work, and spits out a value at the end.

There is one big caveat for all of this, however. Speaking with the folks over at ETH, they use Intel’s Math Kernel Libraries (MKL) for Windows, and they’re seeing some incredible drawbacks. I was told that MKL for Windows doesn’t play well with multiple threads, and as a result any Windows results are going to perform a lot worse than Linux results. On top of that, after a given number of threads (~16), MKL kind of gives up and performance drops of quite substantially.

So why test it at all? Firstly, because we need an AI benchmark, and a bad one is still better than not having one at all. Secondly, if MKL on Windows is the problem, then by publicizing the test, it might just put a boot somewhere for MKL to get fixed. To that end, we’ll stay with the benchmark as long as it remains feasible.

(2-6) AI Benchmark 0.1.2 Total

We saw a few instances where the 35W/45W results were almost identical, with the margin that the 35W would come out ahead in single threaded tasks. This may be because 35W was a fixed setting in the software options, whereas 45W was the power management framework in action.

3D Particle Movement v2.1: Non-AVX and AVX2/AVX512

This is the latest version of this benchmark designed to simulate semi-optimized scientific algorithms taken directly from my doctorate thesis. This involves randomly moving particles in a 3D space using a set of algorithms that define random movement. Version 2.1 improves over 2.0 by passing the main particle structs by reference rather than by value, and decreasing the amount of double->float->double recasts the compiler was adding in.

The initial version of v2.1 is a custom C++ binary of my own code, and flags are in place to allow for multiple loops of the code with a custom benchmark length. By default this version runs six times and outputs the average score to the console, which we capture with a redirection operator that writes to file.

For v2.1, we also have a fully optimized AVX2/AVX512 version, which uses intrinsics to get the best performance out of the software. This was done by a former Intel AVX-512 engineer who now works elsewhere. According to Jim Keller, there are only a couple dozen or so people who understand how to extract the best performance out of a CPU, and this guy is one of them. To keep things honest, AMD also has a copy of the code, but has not proposed any changes.

The 3DPM test is set to output millions of movements per second, rather than time to complete a fixed number of movements.

(2-1) 3D Particle Movement v2.1 (non-AVX)(2-2) 3D Particle Movement v2.1 (Peak AVX)

y-Cruncher 0.78.9506: www.numberworld.org/y-cruncher

If you ask anyone what sort of computer holds the world record for calculating the most digits of pi, I can guarantee that a good portion of those answers might point to some colossus super computer built into a mountain by a super-villain. Fortunately nothing could be further from the truth – the computer with the record is a quad socket Ivy Bridge server with 300 TB of storage. The software that was run to get that was y-cruncher.

Built by Alex Yee over the last part of a decade and some more, y-Cruncher is the software of choice for calculating billions and trillions of digits of the most popular mathematical constants. The software has held the world record for Pi since August 2010, and has broken the record a total of 7 times since. It also holds records for e, the Golden Ratio, and others. According to Alex, the program runs around 500,000 lines of code, and he has multiple binaries each optimized for different families of processors, such as Zen, Ice Lake, Sky Lake, all the way back to Nehalem, using the latest SSE/AVX2/AVX512 instructions where they fit in, and then further optimized for how each core is built.

(2-3) yCruncher 0.78.9506 ST (250m Pi)

For our purposes, we’re calculating Pi, as it is more compute bound than memory bound. In single thread mode we calculate 250 million digits, while in multithreaded mode we go for 2.5 billion digits. That 2.5 billion digit value requires ~12 GB of DRAM, and so is limited to systems with at least 16 GB.

(2-4) yCruncher 0.78.9506 MT (2.5b Pi)

CPU Tests: SPEC Performance CPU Tests: Simulation and Rendering
POST A COMMENT

92 Comments

View All Comments

  • yankeeDDL - Tuesday, March 1, 2022 - link

    Great article, as usual.
    It seems clear that Intel's AL still has the performance advantage, however, in the Conclusion page, the performance comparison is reference to the nominal consumption (35W, 45W, 65W), while we know that Intel's part can reach twice as much power, in practice, making an apples-to-apples comparison quite difficult, especially in light of Intel's better scaling with more Power.

    Is there a way to check the exact performance per core under the same exact consumption (or scaled)?
    I am especially interested as a user of the 1165G, which is an absolute battery eater (and/or heater): it seems that AL is a huge improvement, but if it also draws 100W (instead of 45W) to beat Ryzen by a 10%, then it's not worth it. In my opinion.
    Reply
  • Spunjji - Tuesday, March 1, 2022 - link

    Yes, the overall picture that has built up is of Intel's Alder Lake winning out at higher power levels (40W+) while AMD coming out ahead below that.

    This is good, because it means that we have great options for people who want the best possible performance in a mobile form-factor and for people who want a more even balance of performance and power usage. It's a nicer situation to be in than when Intel complete owned the mobile segment, followed by the years of stagnation at 14nm.
    Reply
  • yankeeDDL - Tuesday, March 1, 2022 - link

    Agree on all points.
    Intel's Tiger Lake is an absolute disaster, and it is actually surprising that Intel only managed to lose 50% market share with such a lousy product compared to Ryzen.
    And equally surprising is the insane jump in performance and perf/watt achieved with AL. Definitely good for the consumers.
    Reply
  • mode_13h - Tuesday, March 1, 2022 - link

    > Intel's Tiger Lake is an absolute disaster

    That seems like an overstatement. It just didn't improve enough against Ryzen, particularly in light of the 5000-series' gains. However, especially in light of Ice Lake's disappointments, Tiger Lake didn't seem so bad.
    Reply
  • Alistair - Tuesday, March 1, 2022 - link

    Tiger Lake was a stroke of luck for Intel, their worst product ever during a massive silicon shortage. They spent the year selling quad cores because AMD was selling everything they could make, not because Tiger Lake was any good. Reply
  • bigboxes - Wednesday, March 2, 2022 - link

    For sure. I went with AMD for the first time since 2006 this last year. Reply
  • Samus - Thursday, March 3, 2022 - link

    The irony here is AMD mobile CPU's are widespread in lots of desktops and AIO's, even high end units. You would rarely, if ever, see Intel U-series parts in desktops\AIO's outside of USFF's or low-end AIO's with Celeron\Pentiums.

    This is happening partially because AMD doesn't have a wide product stack like Intel. And they don't need too. The AMD U-series parts are absolute performance monsters and have been for the last 3 generations.
    Reply
  • abufrejoval - Friday, March 4, 2022 - link

    I own both, a Ryzen 5800U in a notebook and an i7-1165G7 as a NUC.

    They are really quite comparable, both in iGPU performance, in scalar CPU power and even in multi-threaded CPU power.

    At 15 Watts the 8 Ryzen cores operate below the CMOS knee, which means they have to clock so low they can't really gain much against 4 Tiger Lake cores clocking above it. Synthetic benchmarks may prove a lead that's next to impossible to realize or really relevant in day-to-day work. For the heavy lifting, I use a 5950X, which isn't that much faster on scalar loads, but runs almost as many rings around the 5800U as the i7-1165G7: the extra Watts make more of a difference than the cores alone.

    My impression is that the Ryzen needs the higher power envelope, 35 or even 65 Watts, and of course a matching workload to put those extra cores to work. AMD's primary aim for their APUs was to cover as many use cases as possible from a single part and they do amazingly well. If they could afford to do a native 4 core variant as well, I'm pretty sure that would outsell the 8 core.

    In fact the SteamDeck SoC would probably make a better notebook part for many (not everyone).

    And there is nothing wrong with Tiger Lake, except that perhaps today there are better SoCs around: it was and remains a welcome improvement over the previous generations from Intel.

    Buy it used and/or cheaper than these AL parts and you should have little to complain about... unless complaining is what you really enjoy.
    Reply
  • mode_13h - Tuesday, March 1, 2022 - link

    > Great article, as usual.

    I thought so, as well, which was a relief. Then, I noticed the by-line:

    "by Dr. Ian Cutress"
    Reply
  • lemurbutton - Tuesday, March 1, 2022 - link

    People shouldn't care that much about AMD and Intel on laptops right now. M1 series completely destroys both. AMD and Intel are 3-4 years behind. Reply

Log in

Don't have an account? Sign up now