Though it seems like Apple’s hardware divisions can hardly keep a secret these days due to the realities of mass production, the same is fortunately not true for their software divisions. Broad strokes aside, Apple managed to pack in a number of surprises in their OS X and iOS presentations at WWDC yesterday, and there’s nothing that ended up being quite as surprising to me as the announcement of the Metal API for iOS.

Later this week Apple will be holding their Metal developers sessions, at which time we’ll hopefully get some further details on the API and just how Apple intends to have developers use it. In the meantime with the preliminary Metal programming guide posted over on Apple’s developer website, I wanted to spend a few minutes musing over yesterday’s announcement, how Apple ended up developing their own API, and what this may mean for users and game developers.

Why Low-Overhead APIs?

First and foremost, let’s quickly recap just what exactly Apple has announced. Metal is Apple’s forthcoming low-overhead/low-level graphics and compute API for iOS. Metal is primarily geared towards gaming on iOS, and is intended to offer better graphics performance than the existing OpenGL ES API by curtailing driver overhead and giving developers more direct control over the GPU.

As our regular readers are no doubt well aware, Metal is the latest in a wave of low-level graphics APIs to be introduced over the last year in the GPU space, joining the ranks of AMD’s Mantle and Microsoft’s DirectX 12. In the case of Metal, as has been the case of all of these APIs, the idea is rooted in the fact that while high level APIs provide a number of important features from libraries to hardware abstraction, the overhead from this functionality is not worth the benefits, especially in the hands of highly seasoned programmers who have the experience and the means to go close-to-metal and bang on the hardware directly. The situation facing developers in these cases is that at a time when GPU performance growth is rapidly outpacing CPU performance growth, the API and driver overhead has gone from problematic to intolerable, leading to developers wanting to access the hardware directly.


How The Low-Level Mantle API Benefitted DICE's Frostbite Engine

Metal in turn is the API through which Apple will provide this access. By peeling back the driver and API stack to the bare minimum, developers get to tell the GPU exactly what they’re doing and how they want it done, bypassing large chunks of CPU-intensive code that would previously do this for the developer. Whenever we’re talking about these low-level APIs it’s important to note that they’re merely ways to improve efficiency and are not miracle workers, but when faced with the most applicable bottleneck, the draw call – what’s essentially a single function call for the GPU – the increase in throughput can be remarkable. We won’t spend too much more time on the why’s of Metal, as we’ve written much longer outlines on low-level APIs before that don’t need repeated here, but it’s important to establish a baseline for evaluating Metal.

Are SoCs Draw Call Limited?

Upon hearing Apple’s Metal announcement, perhaps the greatest surprise was that iOS developers were in a position where they needed and could benefit from a low-level API like Metal. In the PC space we’ve been seeing low-level APIs rolled out as a solution to the widening gap between CPU and GPU performance, however the SoC class processors in Apple’s iOS devices are a very different beast. As one would expect for a mobile product, neither the CPU nor the GPU is high performance by PC standards, so why should a low-level API be necessary.

The answer to that is that while SoCs are lower performance devices, the same phenomena that has driven low-level APIs on the PC has driven them on mobile devices, just on a smaller scale. GPU performance is outgrowing CPU performance on the SoC level just as it has been the PC level, and even worse, SoC class CPUs are so slow that even small amounts of driver overhead can have a big impact. While we take 4000 draw calls for granted on desktop hardware – overhead and all – it’s something of a sobering reminder that this isn’t possible on even a relatively powerful SoC like the A7 with OpenGL ES, and that it took Metal for Crytek to get that many draw calls in motion, never mind other CPU savings such as precompiled shaders. If Apple intends to further gaming on iOS (and all signs are that they do), then capable programmers are going to want low level GPU access to maximize their graphical quality, the same as they do on the desktop and on game consoles.


Apple Metal Thread Model (Note that no Apple SoC has more than 2 CPU cores yet)

Ecosystems & Portability

But on that note there’s quite a bit that goes into providing developers with these kinds of tools, which puts Apple in a very interesting position among hardware and OS vendors. Of the other low-level APIs we’ve seen so far – AMD’s Mantle and Microsoft’s DirectX 12 – the former is an API established by a hardware vendor who has to ride on top of other companies CPUs and OSes, and the latter is an OS vendor who has to ride on top of third party CPUs and GPUs. Apple on the other hand is in the enviable position of being as close as anyone can be to offering a fully vertical ecosystem. Apple designs their own CPUs, configures their own SoCs, and writes their own OS. The only portion of the chain that Apple doesn’t control is the GPU, and even then the company has exclusively used Imagination Technologies’ PowerVR GPUs for the last 7 years with no signs of this changing. So for all practical purposes Apple has a closed ecosystem that they control from top to bottom, and can design for accordingly.

A closed ecosystem in turn means that Apple can achieve a level of OS, hardware, and programming language integration that no one else can achieve. Metal doesn’t need to take into consideration any other GPU architectures (though Apple in all likelihood has left it generic enough to be portable if the situation arises) and the OS around it can be tailored to the API, rather than making the API fit within the confines of the OS. This doesn’t necessarily mean Apple is going to make significant use of this integration, but it will be interesting to see just what Apple does do with so much control.


A7 SoC Floorplan (Image Courtesy Chipworks)

Another interesting thing to see as Metal plays out is how Apple handles portability from OpenGL ES, that is if they try to handle it at all. On the whole, it’s accepted that a low-level API like Metal will have minimal portability from higher level languages such as OpenGL ES. The exception to this thus far has been that due to the fundamentally low level nature of shader programs, that shader programs have been more portable. In the case of AMD’s Mantle, for example, we have seen AMD specifically support DirectX’s shader language – HLSL – to make porting to Mantle easier. Shader programs are just one part of a bigger picture, but their growing complexity and low level nature means that there are still benefits to being able to port them among APIs even when the API commands themselves are not portable.

At least for the moment, Apple’s Metal programming guide makes no mention of porting from the existing OpenGL ES API. Looking at the Metal shader language and comparing it to the OpenGL ES shader language (GLSL ES), while it’s initially promising since both languages are based on C++, it’s also clear that for better or worse Apple hasn’t held back from eclipsing OpenGL ES here. Metal’s shader language is based on a newer version of C++, C++11, and consequently includes features not available in GLSL ES. Furthermore comparing the function libraries there are again a number of identical functions, and yet more functions that the two shader languages do not have in common. Portability out of Metal aside, it’s not at all clear whether GLSL ES shaders are meaningfully portable into Metal; if they aren’t then that means additional work for developers, a specific concern if Apple is trying to land console-like games for iOS devices. So it will be interesting to see how this plays out.

Of course Android portability is also going to raise a flag here, though at first glance it actually seems unlikely that this will be a concern. Without an equivalent API – and the OpenGL AZDO concept isn’t going to be fully applicable to OpenGL ES – the games that benefit the most from Metal are also the games least likely to be on Android, so while portability from Android looks far from easy, there also appears to be little need to handle it. Android portability would seem to be best handled by traditional porting methods using OpenGL ES, which retains its common API status and will be sufficient for the kinds of games that will run on both ecosystems.

Metal Computing

On a final note, while we’ve discussed graphics almost exclusively thus far, it’s interesting to note that Apple is pitching Metal as an API for GPU compute as well as graphics. Despite being one of the initial promoters of the OpenCL API, Apple has never implemented OpenCL or any other GPU compute API on iOS thus far, even after they adopted the compute-friendly PowerVR Rogue GPU for their A7 SoC. As a result GPU compute on iOS has been limited to what OpenGL ES can be coaxed into, which although not wholly incapable, it is an API designed for dealing with images as opposed to free form data.

The low-level nature of Metal on the other hand means that it’s a good (or at least better) fit for GPU computing, as the lack of abstraction for graphics makes it more capable of handling the workflows and data types of compute tasks. This is one area in particular where the Metal shader language being based on a subset of C++11 is a benefit to Apple, as it provides a solid foundation for writing compute kernels. None the less it remains to be seen just how adaptable Metal is – can it match the compute functionality of OpenCL 1.2 or even OpenGL 4.x compute shaders – but even if it’s only of limited use it means Apple is finally ready to approach GPU computing on iOS devices.

POST A COMMENT

57 Comments

View All Comments

  • Kevin G - Tuesday, June 03, 2014 - link

    The weird thing is that Apple has been hiring lots of GPU hardware engineers from AMD. This is doubly weird considering that Apple is indeed on good terms with Imagination Technologies and extended their contracts for several more years early in 2014. The arrival of Metal only adds another piece to another complex puzzle of what Apple is doing with graphics.

    The other variable with Metal is if it will arrive on the OS X side of things down the road. Apple is a large enough OEM that they can just write the check to Intel, AMD and nVidia to write Metal drivers and they'll do it. The thing is that Apple is still committed to OpenGL on the OS X side of things it seems.
    Reply
  • eanazag - Tuesday, June 03, 2014 - link

    With the A7 SoC Apple could produce an iOS/MacOSX laptop, they just need a compiler (Xcode) that will recompile iOS apps to x86 and OSX apps to ARM [this may be feasible today to an extent]. I think that is the direction they are going. We will likely see their product stack start to blend.

    The GPU compute make more sense from an image & video editing standpoint, which still kinda sucks on iOS. This would be a necessity for an iOS laptop from an Apple customer expectation perspective.
    Reply
  • odaiwai - Wednesday, June 04, 2014 - link

    iOS apps to x86 is easy - they iOS Simulator in XCode does this now. Reply
  • przemo_li - Wednesday, June 04, 2014 - link

    iOS is completely unsuitable for large screens.

    Main roadblock being complete lack of good scaling mechanism for UI.

    Apple solution up to this point was to double screen size. And let the devs quadruple size of their textures/images.

    That wont cut for BIG screens as in 11-13-15 INCH.
    Reply
  • testbug00 - Tuesday, June 03, 2014 - link

    making a GPU. First they licensed everything, than they made the uncore, than they made the CPU, next comes the GPU... Well, actually, fabbing it themself might come first. Reply
  • tipoo - Wednesday, June 04, 2014 - link

    Fabbing it...Themselves? They'd have to buy or build a large fab first, and we haven't heard of either. I think that would come far in the future if ever, custom GPU first. Reply
  • Krysto - Wednesday, June 04, 2014 - link

    If I were Apple, and making my own CPU core, I'd definitely try to control the graphics side, too, which means either buy out Imagination (for a price much smaller than they paid in that ridiculous Beats deal), or make my own GPU.

    That way they can fully control their future. No Intel, no Imagination to mess things up for them. Their chips will be exactly how they want them, when they want them.
    Reply
  • WaltFrench - Wednesday, June 04, 2014 - link

    Trouble with buying Imagination is that (according to the industry info I am looking at) Apple is barely in Imagination's Top 10 customer list, at less than 1% of Imagination sales. These data aren't totally trustworthy, but Apple as a customer is buying expertise that Imagination can afford because of the <b>other</b> 99% of their sales.

    And of course, if Apple bought them, the other customers would need to get the product elsewhere. Worst case, a new competitor, with 100X the sales of the Imagination Division at Apple Inc, would arise; this new, well-funded competitor could well become much more successful.

    So presumably Apple is watching out that another big customer (LG, Marvell & Intel show as its top 3) doesn't snap it up and cut off Apple. Maybe even they have paid for the right to be offered a seat at an auction. But buying Imagination could be more problematic than the current, very successful arrangement.
    Reply
  • richardw115 - Wednesday, June 04, 2014 - link

    According to Imagination Technologies' 2013 annual report, revenue from their largest customer was >33% of total sales. Although not named, that customer is almost certainly Apple. Reply
  • HisDivineOrder - Friday, June 06, 2014 - link

    I think Apple would hire GPU hardware engineers because Apple is always of the belief that in the long term it's better to have the option to make your own than rely on someone else to make something for you.

    That is, to get better prices from Intel, they love to have their own huge CPU design team around they can always point to and say, "So Intel, give us a great deal... or we'll take Macbook Airs to ARM." Boom, better deal.

    "So Imagination, give us a better deal or we'll just make our own GPU." Boom, better deal.

    Haven't you ever noticed how 2-6 months before a major announcement of renewal of a contract with Intel or nVidia/AMD for PC's, you'll start hearing rumors about a major swap? Then suddenly like magic the status quo remains and you'll hear about record profits for Apple's hardware because of "fantastic pricing" given to Apple because of "volume?"

    This is the genius of Tim Cook and this is why Jobs made him CEO. He was/is a master at procuring great pricing on hardware and one of the ways he does that is by ensuring Apple always has a longterm option away from their major source of parts.

    That said, this is also where Apple slipped up with Samsung because they tried that strategy with Samsung before suing them and Samsung shrugged and said, "You need US more than WE need YOU." And by and large, they were right.
    Reply

Log in

Don't have an account? Sign up now