Smartphone Audio Quality Testing
by Chris Heinonen on December 8, 2013 5:15 PM EST- Posted in
- Smartphones
- Audio
- Mobile
- Tablets
- Testing
Maximum Level
The maximum output level is derived from the 1kHz test tone used to determine THD+N. The higher the output from the headphone jack, the louder it can drive a pair of headphones. More importantly, having more power available means when you have dynamic music passages that call for power you are less likely to clip the waveform.
There is no chart to show here, just a number that the Audio Precision gives us. In our test data, the most powerful phone was the iPhone 5, at 32.46 mW of power. Next is the Nexus 5 at 22.24 mW, though we can’t drive it that high. Then the Note 3 at 11.81 mW and finally the Galaxy S4 at 3.895 mW. Doubling the power, from 4 mW to 8 mW, produces a 3 dB increase in volume level. 3 dB is the smallest change in volume levels we can easily hear. So even though the iPhone 5 produces 32 mW vs. 4 mW, that is only a 9 dB difference in volume. 10 dB is doubling the volume, so it isn’t even twice as loud. If you have demanding headphones, you will want as much power as you can get.
Frequency Response
To measure the frequency response we measure a set of 61 tones from 20 Hz to 20 kHz. All of these are then equalized to 1 kHz so we can see the maximum deviation from that level. An ideal phone will be perfectly flat here and allow you to adjust this with an EQ setting, or though your taste in headphones. On this test our best performing phone is the Galaxy S4, as seen here.
The total variation from 1 kHz is only 0.014 dB which is very good. The worst performing phone is the iPhone 5, but its variation is only 0.089 dB.
The iPhone 5 also picked up the 20 kHz tone while the Samsung and many others missed it. If we dropped this tone then it might be just as flat. The iPhone 5 test was run slightly differently, as it can't run the Android test program, which might account for this. For a phone with a different response, here is the HTC One with Beats enabled. Other HTC One testing is still in progress as I write this article.
Here we see that Beats is adding a +3.5 dB boost from 60 Hz to 90 Hz, but the deviation from 0 dB goes from 30Hz to 300 Hz. Past 6.5 kHz we also see a rise in the treble. People often mistake boosted treble for extra detail, which is likely the reasoning behind this. As we see it is far different than the other two examples we looked at.
Facebook
Twitter
RSS
188 Comments
View All Comments
Impulses - Monday, December 9, 2013 - link
If I didn't say this on one of my previous comments I'll say it now, thanks for doing this! Audio quality is far too often ignored, specially amongst the PC/tech enthusiast crowd... And it's rarely tested very objectively when it's talked about at all.I wouldn't mind some subjective impressions atop the objective testing though... Or even some more commentary on Chris' part regarding the data itself. Knowing the Nexus 5 is clipping at max volume is one thing, but the reader might not necessarily realize it's not an issue at lower volume levels etc.
UpSpin - Monday, December 9, 2013 - link
Great review, I'm looking forward to further tests.A few suggestions:
- Such graphs are nice for a detailed analysis but useless for an easy comparison between different smartphones. Find a way to break down the important information in those graphs to one or two numbers, which you then list in a bar diagram to allow a comparison across different devices.
- As you pointed out yourself and also other commenters, testing THD at the max. volume might be industry standard, but it's useless for a normal consumer, especially, again, in such a comparison. As in the display tests where Anandtech adjusts to a fixed display brightness across all devices, you should do the same here. Because different headphones require different volume settings you might chose three settings:
A really silent one (fixed dB), a normal one (fixed dB), and the loudest possible (max. power the device offers).
For example on my HTC Desire I use Sony in-ear ear-buds which I drive with the lowest volume setting possible, else it's simply too loud for me.
- Those three volumes also have the advantage to go in detail in specific areas:
The loudest volume setting can be used to determine how much load the smartphone can drive and the consequences (just what you did with the Nexus 5, excellent).
The normal volume setting is a measurement for overall audio quality across all devices, because that's the one most people will use. So there a focus should be kept on dynamic range, frequency response, distortion, ...
The silent setting is to determine in detail the background noise. On my HTC Desire noise is audible with the Sony in-ear ear-buds (not with lower end normal ones). And as you said, if the device can output a lot of power, naturally the noise to signal ratio becomes smaller. But that's artificial and a useless measurement when compared to other devices. So keep the volume at a uniform low setting and measure the noise to get comparable and meaningful results across all devices. Also make sure to include some noise measurement while in Airplane mode and while transfering some data over mobile. Then you can judge how well the analog part was designed.
Impulses - Monday, December 9, 2013 - link
Problem is, the dB values Chris would use would often not correspond with the same volume (and power output) level you'd use, because you're using different headphones with different sensitivities. Settling on different power levels might be more correct but it'd probably leave a lot of people scratching their heads... It might be worth taking a looksee at the way Tyll @ InnerFidelity tests amps and headphones. He's been doing objective tests for quite a while and has found a pretty good balance as far as how to present the data in an easy to digest manner.stepz - Monday, December 9, 2013 - link
Having a couple different fixed power output comparison points should make not matching your exact listening volume less of an issue. At least it's a less arbitrary measurement point than the point where the manufacturer decided to put a virtual stopper on the volume knob. The fact that max volume clips with a specific set of headphones shouldn't matter too much. If the headphone amp is clipping due to limited current available then using higher impedance headphones may be able to use that volume level without clipping. And if aren't able to use it, just don't crank it up that high and problem solved.supergex - Monday, December 9, 2013 - link
Finally I waiting for this for years.Simple and probably stupid question, will you include Windows Phone smartphone?
Many thanks in advance for these tests.
Osamede - Monday, December 9, 2013 - link
Is this novel - or just getting up to par? From what I recall, GSMArena and Mobile-Review have been doing detailed measurement of phone audio performance for years.tom5 - Monday, December 9, 2013 - link
I had the HTC One and it played much louder than the G2 without distortions on the same headphones. HTC One is in many ways ahead of newer phones like the G2 or Z1.eio - Monday, December 9, 2013 - link
Bravo! you guys always set the standard of a proper product review.eio - Monday, December 9, 2013 - link
It would be even more great if interference can be benchmarked, like the noise of I/O, communication while playing a quiet music.panda-fu - Monday, December 9, 2013 - link
This is a step in a good direction!However, I don't think the "issue" with the Nexus 5 was explained properly. What is happening is that the volume control on the Nexus goes high enough that it's possible for the amplifier to run out of power against the specific load. This is perfectly normal, and the practical implication is that the maximum power of that headphone amplifier is lower than your published measurements - the result should be at <1% THD.
Also, all results should list the load against which they were measured. If you allow massive amounts of distortion and don't specify a load, it would be easy enough to claim that a 100W <1% THD into 8 Ohms rated amplifier is "discovered to be able to produce 1000W" - just as long as you disregard it being driven into a 2 Ohm load with over 50% THD.
A metric that might also be of use, and practical in predicting a headphone amplifier's real-world performance, is output impedance. Smartphones and tablets are usually used with low impedance, sensitive headphones, and if the output impedance is relatively high, it can affect real-world frequency response massively - sometimes similar in scale as the EQ you pointed out in one of the measurements. Of course, providing measurements made against a range of headphones with different impedance characteristics, as you seem to intend to do, will point at the same issue if there is any. But in that case, please provide impedance curves of the reference headphones.