System Performance: UL and BAPCo Benchmarks

Our 2022 Q4 update to the test suite for Windows 11-based systems carries over some of the standard benchmarks we have been using over the last several years. While UL's PCMark makes the list, we have opted to temporarily suspend reporting of BAPCo's SYSmark scores (pending fixture of the energy consumption aspect). Instead, BAPCO's CrossMark multi-platform benchmarking tool has been added to the set along with UL's Procyon suite. While CrossMark employs idle time compression and processes all workloads in an opaque manner, UL's Procyon processes real-world workloads with user interactions (like BAPCo's SYSmark). We have augmented the UL Procyon suite benchmark with our own custom energy measurement setup.

UL PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked select PCs with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark. The scores for each contributing component / use-case environment are also graphed below.

UL PCMark 10 - Performance Scores

The NUC BOX-N97 has a 2W edge in the PL1, and the CPU and iGPU are both clocked slightly higher in it compared to the N100 in the NucBox G2. However, the latter has a slight edge in the memory department with LPDDR5-4800 compared to the DDR4-3200 in the NUC BOX-N97. The results above point to a slight edge for the NUC BOX-N97.

UL Procyon v2.1.544

PCMark 10 utilizes open-source software such as Libre Office and GIMP to evaluate system performance. However, many of their professional benchmark customers have been requesting evaluation with commonly-used commercial software such as Microsoft Office and Adobe applications. In order to serve their needs, UL introduced the Procyon benchmark in late 2020. There are five benchmark categories currently - Office Productivity, AI Inference, Battery Life, Photo Editing, and Video Editing. AI Inference benchmarks are available only for Android devices, while the battery life benchmark is applicable to Windows devices such as notebooks and tablets. We presents results from our processing of the other three benchmarks. Note that Jasper Lake and previous Atom-based systems were not able to complete the UL Procyon benchmarks and are hence not included in the graphs below.

UL Procyon - Office Productivity Scores

The performance of the ADL-N systems are all similar, with the N97 enjoying a slight edge due to the power limit default settings. Having a slightly larger physical footprint allows the system to keep up the power limits at higher values without running into thermal issues.

Unfortunately, in the quest for that bit of extra performance, the NUC BOX-N97 incurs an energy-consumption penalty of 0.96 Wh. This is in comparison to the NucBox G2 for the same Office workloads.

Moving on to the evaluation of Adobe Photoshop and Adobe Lightroom, we find the NUC BOX-N97 enjoying a healthier lead. This is likely due to the extra iGPU power available that probably matters for this workload.

UL Procyon - Photo Editing

The usage of the iGPU for accelerating some of the tasks allows the NUC BOX-N97 to complete the job early while minimizing the energy consumption - a 1.56Wh advantage over the NucBox G2.

UL Procyon evaluates performance for video editing using Adobe Premier Pro.

UL Procyon - Photo Editing

The involvement of the iGPU is a positive for the NUC BOX-N97 again, allowing it to get the top honors by a comfortable margin.

Harnessing the iGPU allows the workload to be completed faster on the NUC BOX-N97 compared to the NucBox G2, and this is reflected again in the energy consumption numbers.

BAPCo CrossMark 1.0.1.86

BAPCo's CrossMark aims to simplify benchmark processing while still delivering scores that roughly tally with SYSmark. The main advantage is the cross-platform nature of the tool - allowing it to be run on smartphones and tablets as well.

BAPCo CrossMark 1.0.1.86 - Sub-Category Scores

The relative performance seen in the UL PCMark 10 workloads also translate to BAPCo Crossmark. Idle time compression doesn't seem to affect the relative ordering, as both the systems adopt the same 25W PL2. The responsiveness of the NUC BOX-N97 is obviously much better than the NucBox G2 because the former uses a PCIe 3.0 x1 NVMe SSD, while the latter uses a SATA SSD. The creativity workload sees the G2 edge ahead by a bit, possibly due to the faster memory.

Setup Notes and Platform Analysis Workstation Performance - SPECworkstation 3.1
Comments Locked

20 Comments

View All Comments

  • markiz - Tuesday, October 17, 2023 - link

    Is support really at all important, or at all, with machines like these?
    I've had a couple of PC over the years and they all lasted 10+ years with no failers of any compononets.
    Nor as a home user have I ever considered BIOS update.
    I imagine in business settings it's much different.
  • hubick - Sunday, October 8, 2023 - link

    "and soldered RAM is not being met with as much derision as before."

    this is where I stopped reading this article.
  • meacupla - Sunday, October 8, 2023 - link

    It's 12GB LPDDR5 on a 6W Celeron. It's fine.
    Even 8GB works fine at 1440p on that CPU.
    4K, you might want 16GB.
  • mode_13h - Monday, October 9, 2023 - link

    > It's 12GB LPDDR5 on a 6W Celeron.

    Did you see the power consumption measurements? These things don't even *idle* at 6W!
  • mode_13h - Sunday, October 8, 2023 - link

    > The key aspect ... is the presence of the 'In-Band ECC' option.
    > ...
    > We confirmed ... that this option would not be removed in a future BIOS version
    > Apparently, Intel has decided to allow In-Band ECC as an official feature of the ... N97.

    Excellent! Thank you!

    If only they had used DDR5, this might be my next mini-PC!

    > MemTest86 hang during memory testing of a known faulty SODIMM
    > instead of reporting errors).

    LOL, wut? I hope that's a Memtest86 bug, and not just what happens when you get memory errors with IB-ECC!
  • ganeshts - Sunday, October 8, 2023 - link

    Faulty SODIMMs can't guarantee only one- or two-bit errors. There could be more.

    The likely explanation for the hang is that IB ECC generates an ECC mismatch interrupt internally to the processor, and MemTest86 is not able to handle it.

    MemTest86 will eventually need to add support for IB ECC interrupt handling / error status readouts.
  • mode_13h - Sunday, October 8, 2023 - link

    Thanks for the review & replying to me.

    > Faulty SODIMMs can't guarantee only one- or two-bit errors. There could be more.

    Okay, I sort of assumed you had some idea of how many errors the DIMM had, like based on doing a memtest scan with ECC disabled. I've seen my share of DIMMs with only a handful of bad cells, so I assumed it was one of those.

    > MemTest86 will eventually need to add support for IB ECC interrupt handling / error status readouts.

    Yes. Sounds like a feature they need to add. Is this the memtest variant from PassMark Software?
  • ganeshts - Monday, October 9, 2023 - link

    Yes, I am using the PassMark variant.

    I have screenshots from mem testing the same SODIMM with the NUCS BOX-1360P/D4 (when they had the IB ECC feature enabled in the BIOS):

    https://www.anandtech.com/show/18732/asrock-indust...
  • mode_13h - Monday, October 9, 2023 - link

    Submit a bug report on it For Great Justice!
  • NextGen_Gamer - Tuesday, October 10, 2023 - link

    @Ganesh T S - I don't think those memory bandwidth figures and specs are correct. GPU-Z is known to not have good support for Intel's lower-end line. I know this personally as an owner of the Intel NUC11ATKPE ("Jasper Lake"). Intel's ark website shows that Alder Lake-N has a single memory channel (64-bit). Running at its fastest, so either DDR5-4800 or LPDDR5-4800, would be 38.4GB/sec of bandwidth. That tracks and make sense with what Intel says their 13th-gen Core series max bandwidth is, being dual-channel DDR5-5600, at 89.6GB/sec.

    Alder Lake-N running in its single-channel DDR4-3200 config then should be only 25.6GB/sec.

Log in

Don't have an account? Sign up now