Snapdragon 800 (MSM8974) Performance Preview: Qualcomm Mobile Development Tablet Tested

115 Comments

View All Comments

• Wilco1 - Wednesday, June 19, 2013 - link

  Having used Apple tablets (with their relatively slow CPU but fast GPUs and hardware accelerated drawing/scrolling), I believe how fast a device feels is more a software/GPU issue than a CPU one. Similarly browsing depends a lot on how fast you can download stuff.
  
  The original ABI press release almost literally reads like an Intel advert. And the chosen benchmark is one of the few where Atom scores well, so that way it not only seems faster then it really is but appears more power efficient as a bonus (as the A15/Kraits have to do more work to get the same score). If you did power numbers on Geekbench then Atom wouldn't look very efficient at all - the only results Intel will show is where they look best.
  
  The Phoronix benchmarks show basically the same as Geekbench: http://www.phoronix.com/scan.php?page=article&...
  
  I have no idea what AnTuTu actually does but Atom scores ridiculously high, so much that either the calculation must be incorrect or there is some kind of cheating going on. For example Atom appears to score more than twice as much as A15 on the memory test, but Geekbench shows a completely different story: Stream - the traditional memory benchmark - shows that A15 is about 3 times faster.
  
  So which is right? Well give me the AnTuTu source code and as a compiler/benchmarking expert I'll explain what is wrong with it and how it could be trivially gamed to show a much higher score (if you look at their web page, it is full of cautions about companies cheating the results). I know for a fact one cannot do this with Stream, which is why I trust the Geekbench results.

• tuxRoller - Wednesday, June 19, 2013 - link

  Where is the code for geekbench? I've been to their site but was unable to find any links.

• Wilco1 - Thursday, June 20, 2013 - link

  I think they want you to pay for it... However just like SPEC they use standard open source benchmarks, so you can find pretty much every one of them if you do a search.

• virtual void - Thursday, June 20, 2013 - link

  I don't really agree that Antutu is the only benchmark where Atom does very well compared to ARM. Look at AnandTechs review of the Samsung Galaxy S4 and notice how well the Motorola Razr i does against the S4 in a number of benchmarks. Razr i uses Z2460 single core Atom, K900 uses Z2580 dual core Atom with 33% faster RAM-interface and WAY faster GPU.
  
  I do own a iPad4, so I know it is a slick device, but by comparing Intel vs ARM SoC I meant when they are running the very similar software. Compare Z2760 on Win8 vs Tegra3 or Krait on Windows RT or Z2460 on Android vs Tegra3/Krait on Android. Z2760 definitely feels faster and Z2460, despite being single core and used in very cheep phones, feels at least on pair with Samsung S3 performance wise on tasks like webbrowsning and usage of non 3D applications (the GPU suck in Z2460 for 3D).
  
  My guess is that applications like a webbrowser and applications like Antutu put a lot more stress on the L2 cache and the RAM-subsystem and Intel just destroys the competition when it comes the L2 latency and compared to ARM, even Atom has decent bandwidth to L2. Geekbench probably runs very close to 100% out of L1 cache.

• Wilco1 - Thursday, June 20, 2013 - link

  The S4 review shows single threaded JavaScript tests, so don't expect core count to make any difference. As I mentioned in another post, Anand shows much lower results for S4 than other sites, so that makes the Razr i seem more competitive.
  
  While Atom and ARM both use Android, Intel does their own builds and modifications for x86. They have a lot of man power so can optimize for the common benchmarks as well as making the user experience better like Apple. I can't comment on your experience but I bet it is not at all related to raw CPU performance.
  
  Geekbench uses the memory system in a serious way, Stream for example measures RAM to RAM copies - if it was L1-L1 copy the scores would be 20 times higher! While Cortex-A9 was behind Atom in L2 latency and some variants had terrible memory performance (ahem Tegra 3), A15 has certainly caught up.
  
  I think it is actually far more likely AnTuTu does some tiny loop that stays entirely within L1 and effectively just measures the frequency of a CPU. They might even multiply single threaded scores by the thread count... Also they seem to be adding up various scores rather than calculating the geometric mean. AnTuTu looks like a broken benchmark.

• virtual void - Thursday, June 20, 2013 - link

  Geekbench might do RAM-to-RAM, but most programs you run on a mobile will have L2 cache hit-rates way over 90%. According to http://www.7-cpu.com/cpu/Cortex-A15.html, Exynos 5250 has a L2 latency of 21 cycles, Saltwell got 15 cycles and Silvermont 13 cycles latency, don't know if I would call 40% higher latency as "caught up"...
  
  AnTuTu tries to simulate "normal" applications like games and other applications by running what they think is "typical" workloads (you can see 3D-scenes and simulated applications as the benchmark runs), but I cannot say I know anything about the details of how this benchmark actually work.
  
  The problem with most benchmarks i.m.h.o is that they focus WAY to much on performance when all cores run at full tilt. Connect ADB to an Android phone and run "top", do some "normal" tasks and check the load-average and CPU-load from the running processes. You rarely see more than one CPU-core being used, I bet you never see more than two cores used at 100%, so the only thing that matters for how fast the device feels is the integer single threaded performance and even the old in-order Atom is competitive here.

• Wilco1 - Friday, June 21, 2013 - link

  The 64KB access time (= fastest L2 access without TLB misses) went from 23 cycles on A9 to 13 cycles on A15, while Atom N2800 does 11. It's clear A15 has caught up even if Atom still has a slightly lower L2 latency.
  
  Yes performance benchmarks will never give an indication of typical use or how responsive a device is. For example how fast software & hardware ramp up clock frequency from a dormant state to active can make a large difference.

• wsw1982 - Thursday, June 20, 2013 - link

  The geekbench seems to me emphasis the kind-of HPC working load. And the AnTuTu is more relative to the mobile usage working load. And the real life mobile user experience is more related to the AnTuTu result than to geekbench result. But anyway, it's really depends on the user case. If some one prefer to use their mobile phone to simulate the nuclear reaction rather than browser the internet or play some game, geekbench is their first choice to judge their phones:) I really admire the develop who gives geekbench it's name:)
  
  And just as a opposite to you, that's why I don't trust the Geekbench result for my mobile choice.

• Wilco1 - Thursday, June 20, 2013 - link

  Geekbench does the traditional benchmarks, so in that way it is a bit like a small variant of SPEC. However what does AnTuTu measure? When I go to their website I cannot find any information on it. All I see is endless announcements about people cheating the scores. That really instills confidence...
  
  Given you seem to have a solid trust in AnTuTu, can you explain in more detail what it actually measures? Have you seen the source code? Is it native code, interpreted or JITted? It seems AnTuTu also does memory, integer and FP scores just like Geekbench, so what makes you claim it is more relevant to mobiles?

• virtual void - Wednesday, June 19, 2013 - link

  One clarification about using Tegra 3 as referents above, Tegra 3 does score higher on Geekbench than Z2670 and way higher than Z2460.