System Performance: UL and BAPCo Benchmarks

Our 2022 Q4 update to the test suite for Windows 11-based systems carries over some of the standard benchmarks we have been using over the last several years, including UL's PCMark. New additions include BAPCo's CrossMark multi-platform benchmarking tool, as well as UL's Procyon benchmark suite.

UL PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked select PCs with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark. The scores for each contributing component / use-case environment are also graphed below.

UL PCMark 10 - Performance Scores

Most of the other systems equipped with the same Core i7-1360P processor manage to outperform the Khadas Mind Premium because their size and cooling solution allows them to configure a PL1 of 40W (compared to the 28W in the Mind Premium). As a result, the system makes its appearance in the second half of the pack.

UL Procyon v2.1.544

PCMark 10 utilizes open-source software such as Libre Office and GIMP to evaluate system performance. However, many of their professional benchmark customers have been requesting evaluation with commonly-used commercial software such as Microsoft Office and Adobe applications. In order to serve their needs, UL introduced the Procyon benchmark in late 2020. There are five benchmark categories currently - Office Productivity, AI Inference, Battery Life, Photo Editing, and Video Editing. AI Inference benchmarks are available only for Android devices, while the battery life benchmark is applicable to Windows devices such as notebooks and tablets. We presents results from our processing of the other three benchmarks.

UL Procyon - Office Productivity Scores

The 28W PL1 configuration dictated by the Mind Premium's form-factor means that the overall ordering is similar to what was observed for the PCMark 10 workloads.

From an energy consumption viewpoint, the Mind Premium gets the job done with an energy consumption of around 10.5 Wh - well above what the Intel NUC (Arena Canyon) with the same processor (albeit, higher PL1) needed.

Moving on to the evaluation of Adobe Photoshop and Adobe Lightroom, we find the PL1 again negatively affecting the position of the Mind Premium, with the configurations being in the bottom half of the graphs.

UL Procyon - Photo Editing

Interestingly, the energy consumption numbers are the best of all RPL-P systems, pointing to the 28W setting being the ideal from an overall energy consumption viewpoint.

UL Procyon evaluates performance for video editing using Adobe Premier Pro.

UL Procyon - Photo Editing

The Mind Premium puts up a much better show here, with only the DDR5-based NUC BOX-1360P/D5 in its 40W avatar notching up a better score among RPL-P systems.

The energy consumption numbers are also reasonable, with the two configurations managing to keep themselves in the top half of the graph.

BAPCo CrossMark 1.0.1.86

BAPCo's CrossMark aims to simplify benchmark processing while still delivering scores that roughly tally with SYSmark. The main advantage is the cross-platform nature of the tool - allowing it to be run on smartphones and tablets as well.

BAPCo CrossMark 1.0.1.86 - Sub-Category Scores

The relative performance seen in the UL benchmarks translate to CrossMark also. This is not entirely unexpected, given the difference in the PL1 / PL2 settings of different systems. The comparison of CrossMark results to those from the other benchmarks above becomes interesting when the systems have similar TDPs but differing PL2 values - in such cases, CrossMark with its idle-time compression methodology may not yield realistic results (where throttling can take place when it wouldn't in the actual use-case). That is not the case here, and so the two Mind Premium configurations end up in the middle of the pack.

Setup Notes and Platform Analysis Workstation Performance - SPECworkstation 3.1
Comments Locked

20 Comments

View All Comments

  • peterfares - Friday, September 15, 2023 - link

    This is really cool, but who is this for?
  • abufrejoval - Friday, September 15, 2023 - link

    I’d say it’s mostly for commuters, who’ll oscillate between two or more workplaces with high frequency and regularity, but don’t want to carry more than the “soul” of the computer with them.

    When I do that, I tend to make that a VM I keep on a high-speed USB stick and I then suspend the VM when I commute. Get’s the job done with a bit of overhead but in a smaller form factor but it means having a physical computer at every workplace and other compromises.

    Having a full-sized GPU dock on every location might be somewhat difficult in terms of budget, but a power primary and somewhat after-hours secondary, might be enough to satisfy a large part of the user base.

    Being able to just pick up the running machine right in the middle of something and then try catching a train or plane running might seem attractive, but Windows tends to glitch in far too many ways to make that realistic.

    I’ve had far too many Windows laptops being woken up from some powersave or even hibernation slumber in the middle of a flight, ostensibly for scheduled maintenance, only to then have them cook themselves and their battery to death for lack of cooling in the onboard luggage: I guess I should be glad they didn’t go as far as combusting, but generally I wound up without a working machine on the busy end of the trip…

    If you own a tiny home, operate in a boat, trailer or some other space constrained place this could be cool, but with an eye on longevity I’d not risk anything that wasn’t standards based and if TB isn’t enough, including dGPU, it’s really just tough luck.

    BTW, I do believe they offer TB and not just USB4, they just don’t have the certification done, because they do mention eGPU on their website.

    Pre-configured only: 32GB is certainly better than 16, even if I’d go for 64, especially at current prices for DRAM. The M.2 slots are only 30mm length so there wasn’t that much variety in terms of updates anyway, but that is currently changing because of these Steam console class devices. I guess the reason they won’t let you open the device is mostly to cut down on customer service issues, because there is just too many people out there who overestimate their technical skills and dexterity.

    I guess mostly it just inspired me into looking using a NUC for this commute style use case instead. They aren’t really that much bigger than this, especially if you don’t carry a power brick. Having to make sure they are properly hibernated, isn’t that much of an issue and they are far more economical.
  • brucethemoose - Friday, September 15, 2023 - link

    > 64-48-48-112 @ 5200

    Whoa, is this a typo?

    The timings seem awfully loose, like waay above default JEDEC.
  • meacupla - Friday, September 15, 2023 - link

    I can't even find what JEDEC specifies for LPDDR5
    Timings for LPDDR have always been looser than their regular counterparts.

    They clock higher at lower voltages, and the timings are loose as a result.
  • Kamen Rider Blade - Friday, September 15, 2023 - link

    So, instead of creating a "Proprietary Standard"?

    Why don't they use the existing PC/104 stacking Board standard that has been around for decades?
  • meacupla - Saturday, September 16, 2023 - link

    PC/104 is meant for internal only. It has exposed and unsupported pins, which makes it easy to bend the pins. It's fragile.
    This slot connector is a more robust design.

    Having said that, oculink and TB4 are plenty robust and have an existing market.
  • Kamen Rider Blade - Saturday, September 16, 2023 - link

    You do know that PC/104 has updated to PCIe/104 and uses PCIe connectors that are plenty strong. Version 3 of the spec has been ratified since Feb 17, 2015.

    Also it would be pretty easy for them to figure out how to create a base board to stack modules onto given the modular nature.

    They could've used a EPIC or EBC MoBo base board and stack modules on top.
  • meacupla - Saturday, September 16, 2023 - link

    not my fault you named the wrong spec
  • sjkpublic@gmail.com - Sunday, September 17, 2023 - link

    One main difference between this and other NUC's is the LPDDR5. This could have been a show stopper if they broke the 64GB barrier. Would consider a 128GB LPDDR5 memory version. Otherwise not much to see here.
  • xol - Wednesday, September 20, 2023 - link

    Polite reminder that "industrial design" means designing a product to be mass produced, not edgelord brushed titanium designed to convinced suckers a product is worth twice its equivalent value

Log in

Don't have an account? Sign up now