After a false start or two, AMD is finally getting the first beta of Mantle out the door. With EA DICE having shipped their Mantle patch for Battlefield 4 and developer Oxide having released their Star Swarm technical demo, the first Mantle-enabled applications have landed. Meanwhile AMD for their part is still hammering out an installation issue on their new Mantle-enabled Catalyst drivers, which has led to them missing their previously scheduled January release date.

In the interim, AMD has released a slightly finickier set of drivers to the press for us to play around with ahead of the public Mantle driver release. These drivers should be functionally and performance identical to the public drivers, they just have an outstanding installation bug that requires a workaround, something that AMD doesn’t want in the shipping version. AMD hasn’t provided a public release date for these drivers – at this point it’s in their best interest to avoid providing release dates they don’t know if they can keep – but given the fact that this is the sole showstopper issue in our press drivers, we certainly don’t expect they’ll take much longer.

In any case, we’re hard at work at the moment putting together our full evaluation of this first version of Mantle. That article won’t be ready until next week, but in the meantime given the immense interest in Mantle, we wanted to quickly publish our first batch of numbers for Battlefield 4. We will have a much wider selection of benchmarks for our full article, including many more video cards and results for Star Swarm, but we wanted to quickly bring you what’s almost certainly going to be the most interesting set of data: Mantle performance with a high-end video card.

For that we’re turning to AMD’s Radeon R9 290X, testing the performance of that card under both Direct3D and Mantle in EA’s Battlefield 4. Battlefield 4 is Mantle’s showcase title and accordingly the first real world use case for AMD’s new API, making it the best place to start. As an application retrofitted with Mantle support we don’t expect Battlefield 4 to tap the complete potential of Mantle right out of the door – certainly not when the Mantle SDK and driver stack itself is still in development – but it can give us an idea of what kind of performance gains we can expect if developers chase the low-hanging fruit offered by Mantle.

What is that low-hanging fruit? For the most part that is going to be CPU bottlenecks, specifically bottlenecking in issuing draw calls. Of all of the bottlenecks that can impact a high performance GPU, keeping it fed can be the biggest bottleneck, and in turn bottlenecking in the draw call submission phase can be the biggest culprit. In the long term Mantle will also benefit GPU performance more directly by optimizing workflows within a GPU, and we already see a small bit of that today in Battlefield 4, but the bulk of the optimizations for these earliest titles have been made around the draw call bottleneck.

For our Mantle preview we’re taking a look at two sections of the Battlefield 4 single player game, the first being from the Tashgar mission and the second being from the South China Sea mission. As was the case with Battlefield 3 the use of single player is less than ideal, but as Battlefield 4 lacks a formal benchmark or for that matter the ability to record multiplayer matches, we’re left with single player if we want to have reasonably repeatable benchmarks. And we’ll definitely want a high degree of repeatability if we’re to be able to distinguish Mantle gains from variability in GPU bound scenarios.

Meanwhile to cover a wider spectrum of possibilities, we’re running our 290X against 3 CPU configurations on our GPU testbed. The first of which is our standard configuration, which is our i7-4960X with all cores and HypterThreading enabled (6C/12T), running at 4.2GHz. Our second configuration drops that down to 4C/4T at 2GHz, to test for the benefits of Mantle on a still relatively large core count at lower clockspeeds. Our final configuration takes the core count down further, to 2C/4T at 3GHz, so that we can see what performance is like for processors with fewer cores but higher clockspeeds.

Finally, on a quick note, for measuring Battlefield 4's performance we're using the game's newly built in PerfOverlay.FrameFileLogEnable feature, which replaces FRAPS in this game due to the fact that FRAPS only works with Direct3D and OpenGL. FrameFileLogEnable logs frame times for later analysis, and from this we can reconstruct the minimum and average framerates, and even the full frame pacing performance of the game (but only from the perspective of the game, not the video card). Today we'll be looking at just the average framerates, but be sure to come back next week for our full evaluation, where we'll have frame pacing data and minimum framerates ready to go.

CPU: Intel Core i7-4960X @ 4.2GHz
Motherboard: ASRock Fatal1ty X79 Professional
Power Supply: Corsair AX1200i
Hard Disk: Samsung SSD 840 EVO (750GB)
Memory: G.Skill RipjawZ DDR3-1866 4 x 8GB (9-10-9-26)
Case: NZXT Phantom 630 Windowed Edition
Monitor: Sharp PN-K321
Video Cards: AMD Radeon R9 290X (Uber)
Video Drivers: AMD Catalyst 14.1 Beta
OS: Windows 8.1 Pro

SP-Tashgar

Our first test comes from the Tashgar mission, and is the benchmark we will be using for day-to-day GPU benchmarking. This benchmark takes place immediately at the start of the mission, with our character driving out of the mountains and into the city of Tashgar. This benchmark has a limited CPU load and is GPU-bound in most situations, which potentially limits the benefits of Mantle in alleviating CPU bottlenecks, but gives us an idea of what kind of performance benefits we can expect in GPU-bound scenarios.

Battlefield 4 - 1080p Ultra - Tashgar - Radeon 290X Uber

Battlefield 4 - 1080p High - Tashgar - Radeon 290X Uber

Battlefield 4 - 1080p Low - Tashgar - Radeon 290X Uber

Battlefield 4 Tashgar: Mantle Performance Gains
  Ultra High Low
i7-4960X 6C/12T @ 4.2GHz 8% 10% -14%
i7-4960X 4C/4T @ 2GHz 8% 13% 26%
i7-4960X 2C/4T @ 3GHz 8% 13% 28%

Even at 1080p Ultra, where the Radeon R9 290X is clearly GPU-bound, we can see that switching to Mantle offers some performance improvements. With our i7-4960X fully powered up, this leads to an 8% performance increase, and we see similar performance increases even with other CPU configurations. Since we don’t appear to be CPU-bound in any appreciable way, this gives us a decent idea of what kind of GPU performance benefits Mantle can offer.

Meanwhile if we switch to High and Low settings, the higher framerates are able to tease out the CPU benefits of Mantle. With BF4’s High settings this is 10-13% depending on the CPU configuration, which indicates we’re still significantly GPU bound here.

Using Low quality settings on the other hand significantly widens the gap in both directions, with the minimum gain being -14%, and the maximum gain being 28%. In the case of our 6C/12T CPU configuration, Mantle actually has a detrimental impact on performance, bringing down our framerate from a positively absurd 216fps to a slightly less absurd 181fps. This was unexpected to say the least, and while we’re not particularly concerned about it given the fact that we have little reason to use this setting in day-to-day gaming, but it does point to a weakness in the current builds of BF4 and the Mantle drivers.

Otherwise if we move to our slower CPU configurations, the benefits are 26% and 28% for 4C/4T and 2C/4T respectively. Despite the fact that the 4C/4T setup has more real cores to work with, which under normal circumstances would be the stronger setup for a highly threaded application, it’s the 2C/4T setup that technically benefits the most. The difference is quite small, but it’s an interesting outcome none the less.

SP-South China Sea

Our second test comes from the South China Sea mission of Battlefield 4, where our character and his squad are on the quickly disintegrating USS Titan. Whereas our first test is rather uniformly GPU-bound, the breakup of the USS Titan offers us the chance to look at a more CPU-bound scenario. Even this scene isn’t exclusively CPU-bound, but with ship parts and other debris flying around everywhere, it’s going to be one of the more strenuous CPU workloads in the single player game.

Battlefield 4 - 1080p Ultra - South China Sea - Radeon 290X Uber

Battlefield 4 - 1080p HIgh - South China Sea - Radeon 290X Uber

Battlefield 4 - 1080p Low - South China Sea - Radeon 290X Uber

Battlefield 4 South China Sea: Mantle Performance Gains
  Ultra High Low
i7-4960X 6C/12T @ 4.2GHz 7% 8% 7%
i7-4960X 4C/4T @ 2GHz 10% 26% 17%
i7-4960X 2C/4T @ 3GHz 10% 30% 28%

Starting once again at 1080p Ultra, even with the greater CPU workload presented by this test, we are unsurprisingly still GPU-bound on Ultra settings. The benefits aren’t as uniform as last time – they now range from 7% to 10% – but it’s safe to say that we’re once again seeing what are mostly the GPU performance benefits of Mantle.

However shifting to High quality shows much greater performance gains, indicating that we’re at least partially (if not fully) CPU-bound here. Once we reduce our CPU performance from 6C/12T to 4C/4T, the performance gains from using Mantle jump from 8% to 26%, and then to 30% when using our 2C/4T configuration. For a game that’s not immensely CPU bound in the first place and has been retrofitted for Mantle, this is towards the upper bound of what we would expect.

Finally switching over to our Low quality settings causes our performance gains to actually taper off some. We’re still CPU-bound on our 4C/4T setup leading to a 17% performance gain, but we’re not as CPU-bound as we were at High quality settings, apparently. Meanwhile the performance gains for 2C/4T remain similar to last time, at 28%. Battlefield 4 has multiple CPU tasks going on here, not the least of which is the simulation itself, so in the case of our 4C/4T setups it’s likely we’ve stumbled onto a situation where the game is more strongly CPU-bound by the simulation and other aspects of the game than it is the submission of draw calls.

First Thoughts

As this is only a brief preview of our results we don’t intend to read too much into this limited data set, but even just looking at the 290X does provide us with some interesting data. For the pure high-end scenario – a 290X or similar GPU with a high-end CPU – Mantle can still offer performance benefits from the GPU workflow optimizations it provides. A 7-10% performance increase is not a dramatic difference, but it is 7-10% better performance than AMD had yesterday.

Meanwhile it comes as little surprise that the greatest performance benefits in our limited BF4 testing come in the mixed performance scenarios, pairing up a high-end GPU with slower CPUs. Since the lowest hanging fruit for Mantle optimizations is going to be CPU draw call bottlenecks, it’s going to be the weaker CPUs that have the most to gain here. In this case we still need to go out of our way to create CPU-bound scenarios – the 290X is rarely held back by the CPU on Ultra quality settings – but when we do create them we can see some of potential that Mantle can offer. At High and Low quality settings, and excluding our one Mantle performance regression, we see performance gains anywhere between 7% and 30%. This shows (if nothing else) that even a retrofit game with a highly optimized Direct3D rendering path, like Battlefield, can still be bottlenecked by draw call performance. And that consequently some of Mantle’s CPU overhead reduction capabilities do in fact pan out.

As for whether all of this is worth the costs and tradeoffs of Mantle from both a consumer perspective and a developer perspective is a longer discussion that we’ll be having next week, alongside our expanded benchmark results. But at first glance it looks like AMD has cleared the first hurdle, which is showcasing that there are tangible benefits to having a low-level graphics API. Now AMD just needs to further hammer out their Mantle drivers and get them into a public-consumable state, so that the wider community of end-users can test and evaluate AMD’s Mantle offering. Outside of the known installation issue we have not encountered any issues with Mantle thus far – this being despite the fact that AMD is being very explicit about the beta nature of the Mantle stack – so hopefully this is a good omen for the company after the delays leading up to this point.


AMD's Official Performance Data

Finally, we’ll quickly close with some of AMD’s performance numbers, which they’ve published in their reviewer’s guide. We feel that vendor-provided should always be taken with a grain of salt, but they do serve their purpose, especially for getting an idea of what performance is like under a best case scenario. To that end we can quickly see that AMD was able to top out at a 41% performance improvement on a 290X paired with an A10-7700K. This is a greater performance gain than the peak gain of 30% we’ve seen in our own results, but not immensely so. More importantly it can give us a good idea of what to reasonably expect for performance under Battlefield 4. If AMD’s results are accurate, then a 40% performance improvement is the most we should be expecting out of Battlefield 4’s Mantle renderer.

Comments Locked

135 Comments

View All Comments

  • Dr_SnM - Saturday, February 1, 2014 - link

    You may have this planned already so I'm sorry if this is telling you something you already know.
    I think it may be quite useful to report the standard deviation (or variance, take your pick) of the frame rates. Averages and minimums may hide an overall smoothing effect which could actually count as a performance increase. In other words, Mantle, by removing CPU bottlenecks, may reduce the frequency of frame rate dips. The best way to quantify this is with a measure of spread such as the standard deviation.

    Thanks guys
  • grndzro77 - Saturday, February 1, 2014 - link

    From what I'v seen the framerate dips are greatly improved.
  • Dr_SnM - Saturday, February 1, 2014 - link

    Yeah, that would be my expectation. Everyone focuses on Average FPS but there is more to a game play experience than that number.
  • chizow - Saturday, February 1, 2014 - link

    Can't see this being worthwhile for devs to implement for a 10% gain on a minority % of hardware (AMD only has 40% of hardware, and only GCN cards are supported). Anyone who has a Intel Quad core since 1st-gen i7 will likely see minimal gains from Mantle and that's before we see how Mantle fares against it's Nvidia counterparts.

    I think the real advantage will be for those who use AMD CPUs, but the chances of someone using an AMD CPU and owning a high-end GCN graphics card are pretty slim based known data sets. Steam for example has Intel at a commanding 74% share with only 24% AMD.
  • Manch - Saturday, February 1, 2014 - link

    But AMD has 100% of the consoles. I know Iknow, Mantle is not what they use on the consoles, but it is similar. Makes porting for the developers a lot easier. Also, relieving that CPU overhead allows developers to now use that freed resource for other things. This is just a quick preview. Wait for the full article.
  • chizow - Saturday, February 1, 2014 - link

    Mantle is as similar to consoles as they are to each other, and as we know, each platform still takes development time and resources. Can't see how it makes porting easier when they have to port 1 more build for the same platform as DirectX is still not going away.

    Relieving CPU overhead is of questionable benefit given many of these games and hardware use cases are already GPU limited, meaning the CPU overhead was never really an issue to begin with. We see significant benefits (>10%) from Mantle only in fringe CPU limited scenarios that are relatively unlikely to begin with (multi-GPU at low resolutions, single GPU at low resolutions/settings).
  • testbug00 - Saturday, February 1, 2014 - link

    Considering Mantle closely resembles PS4 coding and also uses the same GPU as the Xbox One (the coding is probably a bit different due to esram and such) it will take off.

    The big thing is that it costs a very small part of development, about 2-3% from what I have heard, and brings large performance benefits to a noticeable part of the market.

    Also, what counterpart does Nvidia have again?
  • chizow - Saturday, February 1, 2014 - link

    Nvidia's counterpart would be their competing GPUs for each price/performance category, if Nvidia performs as well or better in BF4 and as well or better in other titles, again, is it really a worthwhile endeavor? Would anyone change their buying tendencies for a single game, or a few games per year if it meant crippling performance in the myriad titles that do not use Mantle?

    Mantle resembles PS4 coding as much as PS4 coding resembles XB1 coding, they are all different and as such, take time and resources to produce and validate. The difference of course, is that the PC already has an API that supports 100% of IHVs (AMD, Nvidia, Intel) vs. Mantle which supports just a small fraction (GCN only for a minority AMD share of hardware).

    As for 2-3% being a small part of development? I wouldn't consider that a small part, it just introduces more variables that can go wrong during the development process. We have already seen Mantle was delayed nearly 2 months in BF4 and took nearly 2 years to develop in the first place. For what? 10% gains in most use-case scenarios to a small fraction of the user-base?

    I guess we will see how it goes, but I can't see widespread adoption for these types of gains.
  • djentleman - Sunday, February 2, 2014 - link

    That would only apply if they were only programming for mantle. They still have to program for DX, giving them no incentive except for money to take advantage of it.

    And it really isn't similar to console programming, no developer has confirmed that. That's just internet yap for people who think of themselves as programmers who know.
  • mikato - Wednesday, February 5, 2014 - link

    "giving them no incentive except for money"
    Oh darn, who wants that anyway

Log in

Don't have an account? Sign up now