CPU Tests: Encoding

One of the interesting elements on modern processors is encoding performance. This covers two main areas: encryption/decryption for secure data transfer, and video transcoding from one video format to another.

In the encrypt/decrypt scenario, how data is transferred and by what mechanism is pertinent to on-the-fly encryption of sensitive data - a process by which more modern devices are leaning to for software security.

Video transcoding as a tool to adjust the quality, file size and resolution of a video file has boomed in recent years, such as providing the optimum video for devices before consumption, or for game streamers who are wanting to upload the output from their video camera in real-time. As we move into live 3D video, this task will only get more strenuous, and it turns out that the performance of certain algorithms is a function of the input/output of the content.

HandBrake 1.32: Link

Video transcoding (both encode and decode) is a hot topic in performance metrics as more and more content is being created. First consideration is the standard in which the video is encoded, which can be lossless or lossy, trade performance for file-size, trade quality for file-size, or all of the above can increase encoding rates to help accelerate decoding rates. Alongside Google's favorite codecs, VP9 and AV1, there are others that are prominent: H264, the older codec, is practically everywhere and is designed to be optimized for 1080p video, and HEVC (or H.265) that is aimed to provide the same quality as H264 but at a lower file-size (or better quality for the same size). HEVC is important as 4K is streamed over the air, meaning less bits need to be transferred for the same quality content. There are other codecs coming to market designed for specific use cases all the time.

Handbrake is a favored tool for transcoding, with the later versions using copious amounts of newer APIs to take advantage of co-processors, like GPUs. It is available on Windows via an interface or can be accessed through the command-line, with the latter making our testing easier, with a redirection operator for the console output.

We take the compiled version of this 16-minute YouTube video about Russian CPUs at 1080p30 h264 and convert into three different files: (1) 480p30 ‘Discord’, (2) 720p30 ‘YouTube’, and (3) 4K60 HEVC.

(5-1a) Handbrake 1.3.2, 1080p30 H264 to 480p Discord(5-1b) Handbrake 1.3.2, 1080p30 H264 to 720p YouTube(5-1c) Handbrake 1.3.2, 1080p30 H264 to 4K60 HEVC

Up to the final 4K60 HEVC, in CPU-only mode, the Intel CPU puts up some good gen-on-gen numbers.

7-Zip 1900: Link

The first compression benchmark tool we use is the open-source 7-zip, which typically offers good scaling across multiple cores. 7-zip is the compression tool most cited by readers as one they would rather see benchmarks on, and the program includes a built-in benchmark tool for both compression and decompression.

The tool can either be run from inside the software or through the command line. We take the latter route as it is easier to automate, obtain results, and put through our process. The command line flags available offer an option for repeated runs, and the output provides the average automatically through the console. We direct this output into a text file and regex the required values for compression, decompression, and a combined score.

(5-2c) 7-Zip 1900 Combined Score

AES Encoding

Algorithms using AES coding have spread far and wide as a ubiquitous tool for encryption. Again, this is another CPU limited test, and modern CPUs have special AES pathways to accelerate their performance. We often see scaling in both frequency and cores with this benchmark. We use the latest version of TrueCrypt and run its benchmark mode over 1GB of in-DRAM data. Results shown are the GB/s average of encryption and decryption.

(5-3) AES Encoding

WinRAR 5.90: Link

For the 2020 test suite, we move to the latest version of WinRAR in our compression test. WinRAR in some quarters is more user friendly that 7-Zip, hence its inclusion. Rather than use a benchmark mode as we did with 7-Zip, here we take a set of files representative of a generic stack

  • 33 video files , each 30 seconds, in 1.37 GB,
  • 2834 smaller website files in 370 folders in 150 MB,
  • 100 Beat Saber music tracks and input files, for 451 MB

This is a mixture of compressible and incompressible formats. The results shown are the time taken to encode the file. Due to DRAM caching, we run the test for 20 minutes times and take the average of the last five runs when the benchmark is in a steady state.

For automation, we use AHK’s internal timing tools from initiating the workload until the window closes signifying the end. This means the results are contained within AHK, with an average of the last 5 results being easy enough to calculate.

(5-4) WinRAR 5.90 Test, 3477 files, 1.96 GB

 

CPU Tests: Synthetic

Most of the people in our industry have a love/hate relationship when it comes to synthetic tests. On the one hand, they’re often good for quick summaries of performance and are easy to use, but most of the time the tests aren’t related to any real software. Synthetic tests are often very good at burrowing down to a specific set of instructions and maximizing the performance out of those. Due to requests from a number of our readers, we have the following synthetic tests.

Linux OpenSSL Speed: SHA256

One of our readers reached out in early 2020 and stated that he was interested in looking at OpenSSL hashing rates in Linux. Luckily OpenSSL in Linux has a function called ‘speed’ that allows the user to determine how fast the system is for any given hashing algorithm, as well as signing and verifying messages.

OpenSSL offers a lot of algorithms to choose from, and based on a quick Twitter poll, we narrowed it down to the following:

  1. rsa2048 sign and rsa2048 verify
  2. sha256 at 8K block size
  3. md5 at 8K block size

For each of these tests, we run them in single thread and multithreaded mode. All the graphs are in our benchmark database, Bench, and we use the sha256 results in published reviews.

(8-3c) Linux OpenSSL Speed sha256 8K Block (1T)(8-4c) Linux OpenSSL Speed sha256 8K Block (nT)

Intel comes back into the game in our OpenSSL sha256 test as the AVX512 helps accelerate SHA instructions. It still isn't enough to overcome the dedicated sha256 units inside AMD.

CPU Tests: Legacy and Web

In order to gather data to compare with older benchmarks, we are still keeping a number of tests under our ‘legacy’ section. This includes all the former major versions of CineBench (R15, R11.5, R10) as well as x264 HD 3.0 and the first very naïve version of 3DPM v2.1. We won’t be transferring the data over from the old testing into Bench, otherwise it would be populated with 200 CPUs with only one data point, so it will fill up as we test more CPUs like the others.

The other section here is our web tests.

Web Tests: Kraken, Octane, and Speedometer

Benchmarking using web tools is always a bit difficult. Browsers change almost daily, and the way the web is used changes even quicker. While there is some scope for advanced computational based benchmarks, most users care about responsiveness, which requires a strong back-end to work quickly to provide on the front-end. The benchmarks we chose for our web tests are essentially industry standards – at least once upon a time.

It should be noted that for each test, the browser is closed and re-opened a new with a fresh cache. We use a fixed Chromium version for our tests with the update capabilities removed to ensure consistency.

Mozilla Kraken 1.1

Kraken is a 2010 benchmark from Mozilla and does a series of JavaScript tests. These tests are a little more involved than previous tests, looking at artificial intelligence, audio manipulation, image manipulation, json parsing, and cryptographic functions. The benchmark starts with an initial download of data for the audio and imaging, and then runs through 10 times giving a timed result.

We loop through the 10-run test four times (so that’s a total of 40 runs), and average the four end-results. The result is given as time to complete the test, and we’re reaching a slow asymptotic limit with regards the highest IPC processors.

(7-1) Kraken 1.1 Web Test

Google Octane 2.0

Our second test is also JavaScript based, but uses a lot more variation of newer JS techniques, such as object-oriented programming, kernel simulation, object creation/destruction, garbage collection, array manipulations, compiler latency and code execution.

Octane was developed after the discontinuation of other tests, with the goal of being more web-like than previous tests. It has been a popular benchmark, making it an obvious target for optimizations in the JavaScript engines. Ultimately it was retired in early 2017 due to this, although it is still widely used as a tool to determine general CPU performance in a number of web tasks.

(7-2) Google Octane 2.0 Web Test

Speedometer 2: JavaScript Frameworks

Our newest web test is Speedometer 2, which is a test over a series of JavaScript frameworks to do three simple things: built a list, enable each item in the list, and remove the list. All the frameworks implement the same visual cues, but obviously apply them from different coding angles.

Our test goes through the list of frameworks, and produces a final score indicative of ‘rpm’, one of the benchmarks internal metrics.

We repeat over the benchmark for a dozen loops, taking the average of the last five.

(7-3) Speedometer 2.0 Web Test

Legacy Tests

(6-3a) CineBench R15 ST(6-3b) CineBench R15 MT(6-5a) x264 HD 3.0 Pass 1(6-5b) x264 HD 3.0 Pass 2

CPU Tests: Simulation and Rendering Gaming Tests: Deus Ex Mankind Divided
Comments Locked

279 Comments

View All Comments

  • Makste - Tuesday, April 6, 2021 - link

    I again have to agree with you on this. Especially with the cooler scenario, it is not easy to spot the detail, but you have managed to bring it to the surface. Rocket Lake is not a good upgrade option now that I look at it.
  • Oxford Guy - Wednesday, March 31, 2021 - link

    (Sorry I messed up and forgot quotation marks in the previous post. 1st, 3rd, and 5th paragraphs are quotes from the article.)

    you wrote:
    ‘Rocket Lake on 14nm: The Best of a Bad Situation’

    I fixed it:
    Rocket Lake on 14nm: Intel's Obsolete Node Produces Inferior CPU'

    ‘Intel is promoting that the new Cypress Cove core offers ‘up to a +19%’ instruction per clock (IPC) generational improvement over the cores used in Comet Lake, which are higher frequency variants of Skylake from 2015.’

    What is the performance per watt? What is the performance per decibel? How do those compare with AMD? Performance includes performance per watt and per decibel, whether Intel likes that or not.

    ‘Designing a mass-production silicon layout requires balancing overall die size with expected yields, expected retail costs, required profit margins, and final product performance. Intel could easily make a 20+ core processor with these Cypress Cove cores, however the die size would be too large to be economical, and perhaps the power consumption when all the cores are loaded would necessitate a severe reduction in frequency to keep the power under control. To that end, Intel finalised its design on eight cores.’

    Translation: Intel wanted to maximize margin by feeding us the ‘overclocked few cores’ design paradigm, the same thing AMD did with Radeon VII. It’s a cynical strategy when one has an inferior design. Just like Radeon VII, these run hot, loud, and underperform. AMD banked on enough people irrationally wanting to buy from ‘team red’ to sell those, while its real focus was on peddling Polaris forever™ + consoles in the GPU space. Plus, AMD sells to miners with designs like that one.

    ‘Intel has stated that in the future it will have cores designed for multiple process nodes at the same time, and so given Rocket Lake’s efficiency at the high frequencies, doesn’t this mean the experiment has failed? I say no, because it teaches Intel a lot in how it designs its silicon’

    This is bad spin. This is not an experimental project. This is product being massed produced to be sold to consumers.
  • Oxford Guy - Wednesday, March 31, 2021 - link

    One thing many are missing, with all the debate about AVX-512, is the AVX-2 performance per watt/decibel problem:

    'The rated TDP is 125 W, although we saw 160 W during a regular load, 225 W peaks with an AVX2 rendering load, and 292 W peak power with an AVX-512 compute load'

    Only 225 watts? How much power does AMD's stuff use with equivalent work completion speed?
  • Hifihedgehog - Thursday, April 1, 2021 - link

    "The spin also includes the testing, using a really loud high-CFM CPU cooler in the Intel and a different quieter one on the AMD."

    Keep whining... You'll eventually tire out.

    https://i.imgur.com/HZVC03T.png

    https://i.imgflip.com/53vqce.jpg
  • Makste - Tuesday, April 6, 2021 - link

    Isn't it too much for you to keep posting the same thing over and over?
  • Oxford Guy - Wednesday, March 31, 2021 - link

    Overclocking support page still doesn’t mention that Intel recently discontinued the overclocking warranty, something that was available since Sandy Bridge or something. Why the continued silence on this?

    ‘On the Overclocking Enhancement side of things, this is perhaps where it gets a bit nuanced.’

    How is it an ‘enhancement’ when the chips are already system-melting hot? There isn't much that's nuanced about Intel’s sudden elimination of the overclocking warranty.

    ‘Overall, it’s a performance plus. It makes sense for the users that can also manage the thermals. AMD caught a wind with the feature when it moved to TSMC’s 7nm. I have a feeling that Intel will have to shift to a new manufacturing node to get the best out of ABT’

    It also helps when people use extremely loud very high CFM coolers for their tests. Intel pioneered the giant hidden fridge but deafness-inducing air cooling is another option.

    How much performance will buyers find in the various hearing aids they'll be in the market for? There aren't any good treatments for tinnitus, btw. That's a benefit one gets for life.

    ‘Intel uses one published value for sustained performance, and an unpublished ‘recommended’ value for turbo performance, the latter of which is routinely ignored by motherboard manufacturers.’

    It’s also routinely ignored by Intel since it peddles its deceptive TDP.

    ‘This is showing the full test, and we can see that the higher performance Intel processors do get the job done quicker. However, the AMD Ryzen 7 processor is still the lowest power of them all, and finishes the quickest. By our estimates, the AMD processor is twice as efficient as the Core i9 in this test.’

    Is that with the super-loud very high CFM cooler on the Intel and the smaller weaker Noctua on the AMD? If so, how about a noise comparison? Performance per decibel?

    ‘The cooler we’re using on this test is arguably the best air cooling on the market – a 1.8 kilogram full copper ThermalRight Ultra Extreme, paired with a 170 CFM high static pressure fan from Silverstone.’

    The same publication that kneecapped AMD’s Zen 1 and Zen 2 but refusing to enable XMP for RAM on the very dubious claim that most enthusiasts don’t enter BIOS to switch it on. Most people are going to have that big loud cooler? Does Intel bundle it? Does it provide a coupon? Does the manual say you need cooler from a specific list?
  • BushLin - Wednesday, March 31, 2021 - link

    I won't argue with the rest of your assessment but given these CPUs are essentially factory overclocked close to their limits, the only people who'd benefit from an overclocking warranty are probably a handful of benchmark freaks doing suicide runs on LN2.
  • Oxford Guy - Thursday, April 1, 2021 - link

    That’s why I said the word ‘enhancement’ seems questionable.
  • Oxford Guy - Wednesday, March 31, 2021 - link

    ‘Anyone wanting a new GPU has to actively pay attention to stock levels, or drive to a local store for when a delivery arrives.’

    You forgot the ‘pay the scalper price at retail’ part. MSI, for instance, was the first to raise its prices across the board to Ebay scalper prices and is now threatening to raise them again.

    ‘In a time where we have limited GPUs available, I can very much see users going all out on the CPU/memory side of the equation, perhaps spending a bit extra on the CPU, while they wait for the graphics market to come back into play. After all, who really wants to pay $1300 for an RTX 3070 right now?’

    • That is the worst possible way to deal with planned obsolescence.

    14nm is already obsolete. Now, you’re adding in wating for a very long time to get a GPU, making your already obsolete CPU really obsolete by the time you can get one. If you’re waiting for reasonable prices for GPUs you’re looking at, what, more than a year of waiting?

    ‘Intel’s Rocket Lake as a backported processor design has worked’

    No. It’s a failure. The only reasons Intel will be able to sell it is because AMD is production-constrained and because there isn’t enough competition in the x86 space to force AMD to cut the pricing of the 5000 line.

    Intel also cynically hobbled the CPU by starving it of cores to increase profit for itself, banking that people will buy it anyway. It’s the desktop equivalent of Radeon VII. Small die + way too high clock to ‘compensate’ + too-high price = banking on consumer foolishness to sell them (or mining, in the case of AMD). AVX-512 isn’t really going to sell these like mining sold the Radeon VII.

    ‘However, with the GPU market being so terrible, users could jump an extra $100 and get 50% more AMD cores.’

    No mention of power consumption, heat, and noise. Just ‘cores’ and price tag.
  • Oxford Guy - Wednesday, March 31, 2021 - link

    'Intel could easily make a 20+ core processor with these Cypress Cove cores, however the die size would be too large to be economical'

    Citation needed.

    And, economical for Intel or the customer?

    Besides, going from 8 cores to 20+ is using hyperbole to distract from the facts.

    'and perhaps the power consumption when all the cores are loaded would necessitate a severe reduction in frequency to keep the power under control.'

    The few cores + excessive clocks to 'compensate' strategy is a purely cynical one. It always causes inferior performance per watt. It always causes more noise.

    So, Intel is not only trying to feed us its very obsolete 14nm node, it's trying to do it in the most cynical manner it can: by trying to use 8 cores as the equivalent of what it used to peddle exclusively for the desktop market: quads.

    It thinks it can keep its big margins up by segmenting this much, hoping people will be fooled into thinking the bad performance per watt from too-high clocks is just because of 14nm — not because it's cranking too few cores too high to save itself a few bucks.

    Intel could offer more cores and implement as turbo with a gaming mode that would keep power under control for gaming while maximizing performance. The extra cores would presumably be able to do more work for the watts by keeping clocks/voltage more within the optimal range.

    But no... it would rather give people the illusion of a gaming-optimized part ('8 cores ought to be enough for anyone') when it's only optimized for its margin.

Log in

Don't have an account? Sign up now