Smartphone Audio Quality Testing
by Chris Heinonen on December 8, 2013 5:15 PM EST- Posted in
- Smartphones
- Audio
- Mobile
- Tablets
- Testing
Maximum Level
The maximum output level is derived from the 1kHz test tone used to determine THD+N. The higher the output from the headphone jack, the louder it can drive a pair of headphones. More importantly, having more power available means when you have dynamic music passages that call for power you are less likely to clip the waveform.
There is no chart to show here, just a number that the Audio Precision gives us. In our test data, the most powerful phone was the iPhone 5, at 32.46 mW of power. Next is the Nexus 5 at 22.24 mW, though we can’t drive it that high. Then the Note 3 at 11.81 mW and finally the Galaxy S4 at 3.895 mW. Doubling the power, from 4 mW to 8 mW, produces a 3 dB increase in volume level. 3 dB is the smallest change in volume levels we can easily hear. So even though the iPhone 5 produces 32 mW vs. 4 mW, that is only a 9 dB difference in volume. 10 dB is doubling the volume, so it isn’t even twice as loud. If you have demanding headphones, you will want as much power as you can get.
Frequency Response
To measure the frequency response we measure a set of 61 tones from 20 Hz to 20 kHz. All of these are then equalized to 1 kHz so we can see the maximum deviation from that level. An ideal phone will be perfectly flat here and allow you to adjust this with an EQ setting, or though your taste in headphones. On this test our best performing phone is the Galaxy S4, as seen here.
The total variation from 1 kHz is only 0.014 dB which is very good. The worst performing phone is the iPhone 5, but its variation is only 0.089 dB.
The iPhone 5 also picked up the 20 kHz tone while the Samsung and many others missed it. If we dropped this tone then it might be just as flat. The iPhone 5 test was run slightly differently, as it can't run the Android test program, which might account for this. For a phone with a different response, here is the HTC One with Beats enabled. Other HTC One testing is still in progress as I write this article.
Here we see that Beats is adding a +3.5 dB boost from 60 Hz to 90 Hz, but the deviation from 0 dB goes from 30Hz to 300 Hz. Past 6.5 kHz we also see a rise in the treble. People often mistake boosted treble for extra detail, which is likely the reasoning behind this. As we see it is far different than the other two examples we looked at.
Facebook
Twitter
RSS
188 Comments
View All Comments
DanNeely - Sunday, December 8, 2013 - link
I'm curious how well your Grado headphone's are holding up? I bought a pair of SR80's for use at work last winter; but the wire started to develop damage a month or two ago. If I move the wrong way while wearing them I can get brief bursts of static in one ear, and can mute that ear by pinching the cable just above the Y. I suspect the damage was caused by the post in the headband allowing the earcups to spin freely, combined with the unmarked cable making it hard to notice anything less than a half dozen or so revolutions of twist. I'm wondering how much of this is bad luck on my part vs poor design/manufacturing.Marovincian - Monday, December 9, 2013 - link
I had similiar concens with my Grados (actually mine are Allesandro ms1i). I sent grado an email suggesting that they put a stripe on the "Y" wires so that you could more easily straighten them out. They said that they would pass it along to their design team. Then they sent me a free T-Shirt. Classy company for sure.ManuLM - Sunday, December 8, 2013 - link
Quite a good initiative thanks, it is too hard to get these numbers nowadays.I would suggest you guys build up a database over time of phones performance (see headphoneinfo awesome job for instance).
I also suggest that you add to your test the maximum output delivered (power or voltage swing into load). This is interesting, because if a phone clips at high volumes, but its output power is 10dB above the others in average, then the normal user will simply not see the drawback (altough I admit this is initially poor job from the company in tuning the audio system).
It also helps to chase the brands which deliver lower output power, that can turn to a problem on more demanding headphones (high impedance requiring higher voltage swing). Some users will fancy some extra power on their headphone output (even if this might not be safe for their ears).
Last point, some high-end IEMs have quite low impedance, that demand fairly high current specially in the high energy low frequency, creating bass roll off. A simple frequency response check on a low impedance IEM would show this.
RandomUsername3245 - Sunday, December 8, 2013 - link
I like the idea of audio testing, but I am disappointed by the methods used in this article: why would you bother testing a device at maximum volume when you know it is clipping badly? You should reduce the volume to a setting where it does not clip and then continue the review. You can then report the maximum useable volume setting on the device.The maximum volume on an iPhone is reported to be in excess of 100 dB. Listening at this volume for even a short period (15 minutes) on a consistent basis will permanently damage your hearing. Why not test these devices at reasonable volume levels?
http://www.theatlantic.com/health/archive/2011/12/...
(hopefully not too flawed analogy follows...)
If you are comparing two overclocked computers for maximum performance, you set them to their highest stable clock rate and then benchmark. You do not set one to a clock rate that causes continual crashing, and then report that it failed several of the benchmarks. I think this is comparable to audio review for the clipping cellphones. You might argue that the device should support any user-accessible volume level, but historically it is very common for audio amplifiers to allow users to adjust the gain until the output clips. Apple is an unusual case that limits the user to only access non-clipping gain settings.
ManuLM - Sunday, December 8, 2013 - link
audio systems are tested at max performance (there are many reasons for that, including the fact that when you sell something, all usage range of the system should be good), so analogy with OC is not ideal.I agree with you though that testing at nominal volume could help, as an adder only of max volume testing
eio - Monday, December 9, 2013 - link
yes, power of drive is a good factor in a benchmark. but performances at different loads should not be compared directly.eio - Monday, December 9, 2013 - link
a ideal test may have several series of performance graphs with several steps of incrementing loads...RandomUsername3245 - Wednesday, December 11, 2013 - link
Late reply...Like I said in my previous comment, it is common for audio amplifiers to allow you to adjust the gain past where the amplifier will start to clip. You should never expect a car stereo or home theater amplifier to allow you to run at maximum gain without clipping, so why should you expect a phone's headphone amplifier to behave differently?
The proper way to run this test is to adjust the amplifier to maximum non-clipped gain and then run the test.
willis936 - Sunday, December 8, 2013 - link
The day has finally arrived? Good data with some surprising results. I think I'm mostly surprised at how well all of the devices perform. I think dynamic range is perhaps the most important test here simply because most people won't be listening at max volume on headphones and pushing the noise floor down as low as possible is important for quiet listening.Were these tests done on the AKG K701? That is well known as a difficult to drive pair of cans without an amp. If a phone can drive those loudly with good measurements then it's certainly good enough for anything I'd use it for. Testing should be done worst case and if there's time more typical cases. When using my phone as a line out I'll typically leave it 3 steps below max because I expected there to be output stage power issues (seen as dramatic clipping on LG's stuff :x) on my phone. Any lower and as you noted the static noise floor lowers the SNR.
I was a little surprised at the weak channel separation in the otherwise amazing iphone. Channel separation is already a p big issue. Even with expensive headphones it's easy to test and ballpark a crosstalk of worse than -60dB by ear just from the jack to the drivers.
I'd like to make a request for some data of testing devices (1 iphone and 1 iconic android per year?) going backwards to see a progression (or maybe lack thereof) of audio quality in smartphones over the past 4 or 5 years.
willis936 - Sunday, December 8, 2013 - link
Oh, and thanks for the excellent write up and all of your hard work! I'm looking forward to future data.