Retesting the ASUS Rampage IV Black Edition Audio and the Pitfalls of Audio Testing
by Ian Cutress on January 31, 2014 4:45 AM EST- Posted in
- Motherboards
- Asus
- Audio
- Audio Precision
In the motherboard section of AnandTech, we rarely touched the audio section of the motherboard until fairly recently. Audio is a delicate issue, and knowledge on the subject falls into a wide range from the tone deaf all the way to the hyper enthusiast. Professional and prosumer audio equipment can be honed to perfection and cost an appropriate amount, where discussion on quality over $10k per meter cables can happen. It is very hard to quantify an audio experience into a single set of rules, but as the motherboard arena can feature rather elaborate audio setups designed to enhance the quality or provide sufficient electrical filtering, and that these can be some of the major selling points, several of our readers asked us to start testing the audio.
To put this into perspective, a motherboard manufacturer can create an audio portion of the motherboard that costs $2 to buy and implement. Another manufacturer will spend $20 to get a better audio codec and DAC, better audio filters, headphone amplifiers, and specifically separate the digital and audio traces on the motherboard as to reduce potential sources of interference, as well as focusing on a single output to achieve all this. Part of testing these scenarios is producing such a test that a reader can relate to, as well as a series of numbers for comparison to put in the review – both manufacturers and readers want to see that their hard earned engineering money and effort went into something that can be compared.
In light of this request to test audio, looking into the best way to test it is a minefield. The correct equipment in order to get an accurate test of dynamic range, noise and distortion costs over $10k – for example, the audio equipment used in Chris Heinonen’s recent foray into smartphone audio testing was an Audio Precision APx582, which starts at eleven thousand. Audio Precision is often cited as the company that all audio manufacturers go to in order to test their hardware. Unfortunately unlike Chris I am not able to source Audio Precision testing equipment due to my location and contacts, but having the hardware on hand takes everything out of the equation except the audio generator and filtering pathway that a manufacturer decides to use.
One ‘quick and easy’ test that has been in the mix and in several press pack guides is Rightmark: Audio Analyzer (RM:AA). This is a software solution to what is ultimately a hardware problem, and relies on the same hardware doing both the signal production and the signal receiving. This causes a number of issues, such as channel leakage, harmonic resonance in adjacent signals and that the limiting factor of the hardware being testing might affect other parts of the test. More often than not it is also difficult to compare results between products, due to input levels and driver settings. In order to provide accurate and comparable results, the software has to be explicitly set up for consistency and comparison, something a lot of reviewers that use RM:AA do not do.
I must confess that for a while I was in this crowd. My RM:AA test was ultimately out of the box and produced wildly conflicting results. When audio solutions were quoted as 108 dB for signal-to-noise ratio were getting 92 dB, the conclusions were along the lines of differences in signaling and manufacture of the motherboard. In reality, the test was very wrong – the OS by default limits input/output to 16-bit + 44.1 kHz, producing an artificial ceiling, and the volume ratio of output/input in the testing loop was not standardized. While I kept the input/output the same, in fact each output of the audio jacks can have different responses to the audio in question.
After delving further into the topic, we standardized the RM:AA test at AnandTech. The OS is now set to 24-bit, 192 kHz input and output (or the highest possible based on the audio codec options), the output volume was set to 100% and the input volume adjusted to provide accurate results without over-distorting the input. Ultimately our testing is still limited to the level of the input on the codec, often cited around 104 dB SNR compared to a 108 dB or 112 dB peak output, depending on what audio codec + DAC are used. Even when the same 3.5mm shielded cable is used or different tests, cable losses in a uncontrolled environment can be excessive (1-2 dB SNR), and my location next to a busy main road might also introduce error.
This test is in no way perfect – we typically quote the SNR and THD+N (Total Harmonic Distortion + Noise) results as these were ultimately relevant to those who wanted cleaner and louder output, rather than a quality control test.
Recently we tested the ASUS Rampage IV Black Edition. In this review, we noted that for ASUS’ audio solution, they used their SupremeFX technology – a shielded audio codec with separated tracing, audio filters, headphone amplifiers and gold plated connectors. In this, ASUS use a Realtek audio codec to power a Cirrus Logic DAC, rated at 120 dB SNR and -107 dB THD+N. Our results were well below this in terms of SNR, ~95 dB, which is substantially lower. At the time I was unsure of the reason, and needless to say ASUS got back to me to confirm my testing methodology.
After retests on my end, and a more professional test from ASUS, we exchanged new numbers for comparison. My initial numbers were indeed erroneous. After retesting, the ASUS Rampage IV Black Edition falls in the middle of the pack:
ASUS and I compared our SNR graphs to see the differences:
ASUS achieve a -108 dB SNR compared to my -101 dB SNR result due to ASUS using Audio Precision testing tools and my test using the Line In + RMAA methodology. The RMAA test also relies on a 1 kHz pulse at -60 dB for its test rather than a range of frequencies, which as a result causes the peak seen at 1 kHz.
In light of the discrepancy, we are coming to a fork in the road for audio testing. I fall wholeheartedly into the ‘tone deaf’ part of the crowd when it comes to analyzing audio and prefer an easy-to-read number as a comparison, although I understand this is not the be-all and end-all when it comes to audio. Ideally I would like to use Chris’ setup for future motherboard tests. I have asked Chris to perform testing on a few select models if he has time in the future, which might be in the works. But I do not have access to an AP testing setup, and my current solution sits with RM:AA. ASUS has offered to provide their results in future press packs, although independent verification is what we want to report. In light of motherboard development we may see other manufacturers doing it also. In order to improve our reporting in this area, I will in future publish the SNR graphs we receive from RM:AA for comparison.
Facebook
Twitter
RSS
59 Comments
View All Comments
TruePath - Saturday, February 1, 2014 - link
This business about wanting a warm sound and the like made sense when we were stuck in the bad old days of analog audio but it shouldn't persist.Deliberately choosing an audio component that distorts the sound being reproduced only ensures that you can never get exactly the sound you want. Sure, it might make the sound seem more warm but when you want to play something where the input already captures that warm sound you still get that distortion on top of the already warm audio you are playing.
Now that audio files are all digital instead of choosing an audio solution less capable of reproducing the recorded sound the most accurately (I know that's a bit vague) simply reprocessing the sounds in software should suffice to imbue them with whatever effect you desire. This then would allow you to take the same audio files from one device to another and expect them to all sound very very similar.
Yes, I know that if you want your in game sound to be warm this isn't really an option but really, how good can the game be, if you are paying attention to the fact that the background music isn't as warm as you usually prefer. Ideally, you could tune these parameters in hardware (and at the high end I'm sure you can) so you could get the effect even in games.
TruePath - Saturday, February 1, 2014 - link
Numbers can't be an inaccurate substitute for ears. Our machines, which represent the audio in numbers, are far far more sensitive to changes in pressure (sound) than the human ear.I don't doubt that some people with sensitive ears get something out of some reasonably subtle effects. However, just as with wine when double blind studies are done many people are not able to reliably identify which of the experiences is better despite claiming it was clear as day when they knew what was what.
The suggestion that you just go into the store and see what you like is a recipe for wasting money. You know that people are easily subject to placebo effects here and yet you place yourself in a situation where a friendly sales guy can subtly maneuver you using your desire to appear sophisticated and knowledgeable into convincing yourself that, why yes, you can tell the difference between the cheap junk unit and the fancy one.
Also, as I've already suggested there is absolutely no need to go out and buy an audio unit tuned to your tastes. It should be possible to simply buy the one that objective analysis indicates is best able (using taste only to decide what kind of distortions matter more) to reproduce the widest range of sound at your ears and simply modify the audio in software so that the actual sound you hear is whatever you prefer.
ThreeDee912 - Friday, January 31, 2014 - link
Unfortunately while a good portion of audio does involve actual science, there is a large amount of pseudo-science, subjective opinion, and even placebo effect involved. A couple years ago, there was a bit of a debate about lossless 24-bit 192kHz audio being better than normal 16-bit 44.1 kHz, and how people with "golden ears" or "better ears" could clearly hear the larger range of frequencies. Then some people at Xiph, who develop the popular FLAC audio codec, stepped in and showed how they were all being silly. I particularly like how they compared these audiophiles to hypothetical "spectrophiles" that claim they can see X-rays and microwaves everywhere. It just doesn't happen.http://xiph.org/~xiphmont/demo/neil-young.html
Even earlier than that, years before iTunes came around, there were 2 competing MP3 player programs for Mac, Audion and SoundJam. Despite both of them implementing the MP3 decoder the same way, numerous audiophile review websites and magazines all claimed that Audion somehow sounded "better" than SoundJam, even though the Audion devs weren't doing anything special. It's also mentioned in an article about the history of Audion and iTunes on the dev's site.
https://www.panic.com/extras/audionstory/
sheh - Saturday, February 1, 2014 - link
I'm not familiar with these players, but MP3 decoders aren't necessarily created the same (e.g., dithering, among other things), and the rest of the playback chain can be variable as well.Jedibeeftrix - Tuesday, February 11, 2014 - link
brilliant comment. +1 for you, sir.davidedney123 - Friday, January 31, 2014 - link
I really feel all these advanced audio options on motherboards are pure marketing. If you give a damn about audio quality from your PC you use a digital output and an external DAC, and these days there are lots of very cheap options available. All of the "audiophile" components in the world are not going to give you good results if they are located in the EM quagmire inside a PC.willis936 - Friday, January 31, 2014 - link
While it's a hard pill to swallow with all of the marketing fluff that surrounds audio and the even more flamboyant marketing around PC OEMs and motherboards there is good engineering sense behind the things they do. "THE RED LINE". Cheesy as hell but having a separate ground plane helps and paying close attention to the ground path of every component is absolutely necessary in high fidelity audio circuits. There's no shortage of reading on the characteristic of caps and in some places the quality of the cap won't affect performance but in other places close to the signal path the can. Nichicon, WIMA, Nippon, etc. are all more expensive but also _may_ result in better performance.The marketing gimmicks alone don't make a good product though. The implimentation (ie the layout) is much more important than the components chosen. If you pull up the spec sheet for any one of these codecs you'll see absurd SNR and dynamic range numbers. Expecting those out of your speakers is like expecting to get 1.9Gbps out of your fancy 11ac router. It's just not going to happen for practical reasons. The quality of the tiny circuit around the codec is going to be largest limiting factor in most cases.
So while it may seem like marketing trash there's a good reason attention is being paid to this part of the board. There is absolutely no reason at this day and age to spend $200 on a dedicated sound card unless you are working in a professional studio. Even these fancy on board sound cards are good enough for amateur recording given a $100 mic.
thewhat - Friday, January 31, 2014 - link
I appreciate the RMAA tests that you're doing. Even if it has it's limits, it's a very useful tool, IMO.But it seems that, when you're testing, you're always using the same onboard soundcard for the input. As you also noted, this means that the input performance can "bottleneck" the results. But I'm guessing the majority of readers don't use the line-in at all and would rather just see the clear line-out performance and the headphone performance (you can test the latter with a 3.5mm splitter and a headphone load).
RMAA lets you use different in/out devices, so why not get a good quality dedicated soundcard that has a verified good quality input with high SNR and use that for testing the output on the motherboards? (And you could also use that to test the input performance independently.)
mr_tawan - Friday, January 31, 2014 - link
I second that. You could get a good USB audio interface to do the test. This way even though the bottleneck would be possed on the audio interface, at least the test runs with consistency across test subject (mainboard).The problem would be to get the proper audio interface to work with.
PS. Audio Interface is another name of the device we call sound card, usually used for professional-level AFAIK.
bludragon - Friday, January 31, 2014 - link
THIS. At least then you keep the variable quality of the input fixed when comparing outputs and you can also run things the other way - have the external reference provide the output to test the input on the motherboard. There are plenty of "pro" audio usb options you could use for this for not too much $$. e.g. E-MU 0204-USB