Rightware, makers of MobileMark, BrowserMark, and other mobile benchmarking tools recently launched a new 3D benchmark for mobile devices named BaseMark GUI. BaseMark joins our growing suite of mobile device benchmarks and tests OpenGL ES performance on Android and other platforms with an emphasis on 3D user interfaces. We've run BaseMark on almost all the devices we've got on hand, and will do so going forward on other devices we get our hands on.

We've been making a lot of additions to our mobile benchmarking suite lately in an effort to get the best picture possible of the performance landscape out there for tablets and smartphones alike. First it was GLBenchmark 2.0, now it's Basemark. This new test benchmarks OpenGL ES 1.0 or 2.0 performance on devices through a suite of five feature tests and one overall 3D user interface test. In its current incarnation, we've been given binaries for the Android platform, however Basemark's Kanzi 2.0 engine, which is written entirely in ANSI-C, will be ported to other platforms in the future. For now however, we're focusing on testing Android hardware with Basemark and using the OpenGL ES 2.0 render pipelines since all of the Android devices we have (and coming in the future) are of the 2.0 sort.

Basemark divides benchmarking into two categories - feature tests and the final overall user interface test. Up first are the feature tests. These are named animation, vertex streaming, texture streaming, and alpha blending as shown in the menu above. Each tests a different collection of what Rightware believes are representative graphics tasks that will be leveraged in user interfaces moving forward. The first item in that test menu screenshot above - 3D UI - is an overall graphics performance test designed to simulate a completely 3D user interface with example application interfaces. It's one of the heaviest 3D benchmarks I've seen yet. 

First up among those feature tests is animation, which animates a graphics element (in this case, a robot moving through a set of actions) by stepping through a table of keyframes and interpolating the character's movement between using splines. This particular benchmark uses ES 2.0 APIs and per vertex lighting, but the purpose of this test is to be as CPU-bound as possible and specifically stress floating point performance.


Animation Test

Next are the vertex and texture streaming tests, both of which measure asset streaming performance - according to RightWare, effectively memory bandwidth. The vertex test loads geometry data into GPU memory and frees it when no longer needed. The scene itself involves a lot of geometry - ten repeating city blocks which the camera moves through with increasing speed and draw distance. The test ramps from around 3k vertices to 15k vertices per frame, and 190k to 250k triangles per frame. There's a single texture material, fog, two lights, and OpenGL ES 2.0 shaders which use per vertex lighting.


Vertex Test

The texture test is a bit more straightforward, quickly loading images from RAM into the GPU memory and discarding them.


Texture Test

These asset streaming tests effectively test memory bandwidth from disk into RAM and into to GPU memory. Although Kanzi supports ETC texture compression (and Basemark exposes this option for testing), we've initially tested with the default configuration which has ETC texture compression disabled. 

Blend testing - as its name implies - tests alpha blended rendering by drawing a number of semi-transparent contact cards atop each other. These are overlaid sequentially until we reach a desired number of layers. This test actually runs twice, first with front to back ordering of these contact cards, and then with back to front ordering.

This ordering difference shouldn't be confused with the fact that feature test actually runs in both back to front and front to back rendering orders and are combined later.


Blend Test

Next is the composition test, which composites interface screens rendered off-scene into one large tiled user interface. The example here are applications rendering in the background being composited into one view. This test combines 16 off-scene objects. 


Composition Test

There's also a battery test which loops the composition test indefinitely until battery dies. Going forward we will likely use Basemark's battery test to augment our three other ongoing battery test metholodogies. 


3D UI Test

Last but not least is an overall 3D performance test which emulates a futuristic interface consisting of applications rendered on screens attached to a central... orb. This is definitely the most challenging of the Basemark suite, combining post processing blur for depth of field and basic blur, per pixel lighting with three point lights and one directional light, cube and normal mapping, ambient occlusion mapping, and a shader on the arms holding the displays around the cube which animates a texture map. To start we ran with 4xAA enabled (by Basemark default), though we're running Basemark GUI with AA turned off as well. Remember for a moment that all of the phones we've seen so far are tile-based renderers. 

Screenshots and Performance Results
POST A COMMENT

20 Comments

View All Comments

  • QChronoD - Tuesday, January 11, 2011 - link

    Cool article, it's amazing how fast cell phones are getting now.
    Are they planning on releasing a version that can be run on iOS?

    I know that this will probably start a flame war, but I think that's really what Android phones are competing against. There is competition between the different models, but choosing which android phone seems to be a secondary decision, after you've already made the choice not to get an iPhone.
    (has an iPhone4 and loves it, has played with a few android tablets but wasn't all that impressed with the UI)

    BTW, the music in the video sounds like its from some bad Cinemax soft-core porno
    Reply
  • alovell83 - Tuesday, January 11, 2011 - link

    Comparing vs the current iPhone wouldn't really be fair to the iPhone due to the processor requiring more power to push the higher resolution.

    So, I'm not flamming when I say that I'm not impressed with the iPad. I haven't played with a Galaxy Tab, but for my usage other than video/gaming, I know the Galaxy Tab is better for me. I have a Galaxy S phone and the browsing experience is just better, no page refreshes and flash are really big for me. I hate, hate, hate when I open too many web pages in the iPad and it then proceeds to refresh every page that I put into the foreground. Or, when I leave the Safari Browser to do something else and it has to refresh the page.

    The first, pre-HC Android tablets didn't have much to offer anyone looking to be impressed. But, Apple has a serious hole to fill if it wants to catch up to HC in terms of browsing, UI, and so many areas other than apps.
    Reply
  • marraco - Tuesday, January 11, 2011 - link

    Is unfair to cheat the consumer into thinking that by buying a larger screen, he will have the same experience, but on large screen.

    Who sell the large screen should warn the consumer that it is not completely functional, because do not have all the hardware necessary to use the large screen.
    Reply
  • MySchizoBuddy - Tuesday, January 11, 2011 - link

    Galaxy Tab review a Pocketable train wreck on Gizmodo

    http://gizmodo.com/5686161/samsung-galaxy-tab-revi...

    Quotes
    This thing is just a mess. It's like a tablet drunkenly hooked up with a phone, and then took the fetus swimming in a Superfund cleanup site. The browser is miserable, at least when Flash is enabled. It goes catatonic, scrolling is laggy, and it can get laughably bad.
    Reply
  • abrar - Tuesday, January 11, 2011 - link

    isnt it available for public use ? i wanted to test my device ;) Reply
  • abrar - Tuesday, January 11, 2011 - link

    How can one subscribe to comments in anandt. articles ? Reply
  • ltcommanderdata - Tuesday, January 11, 2011 - link

    In terms of real world usage, I thought Android`s GUI performance was more limited by lack of implementation of GPU acceleration rather than lack of performance on the part of GPUs. As such, is this GUI benchmark actually able to tell you what Android phone will feel faster/smoother in the UI?

    I`m guessing despite the GUI name of the benchmark, it`s still more of a measure of hypothetical raw GPU performance than GUI performance experienced by the end-user.
    Reply
  • Brian Klug - Tuesday, January 11, 2011 - link

    "It`s still more of a measure of hypothetical raw GPU performance than GUI performance experienced by the end-user."

    That's actually exactly how I would characterize BaseMark GUI. RightWare's using "GUI" in the name because they're using their own Kanzi tool's UIs (which they believe are similar to possible future 3D user interfaces), but to that extent it's primarily a 3D benchmark which is why we're interested.

    -Brian
    Reply
  • TareX - Tuesday, January 11, 2011 - link

    Things are looking good for the PSP phone, which has the G2's same SoC.... Reply
  • thornburg - Tuesday, January 11, 2011 - link

    The distinct lack of Verizon's "Droid" phones, such as Droid 2 and Droid X is a bit disturbing.

    In fact, there are no Verizon phones at all included in this. Why is that? Actually, aren't these phones only from Sprint and T-Mobile?

    Some kind of carrier bias by AT?
    Reply

Log in

Don't have an account? Sign up now