The A13's Memory Subsystem: Faster L2, More SLC BW

The memory subsystem of a chip is an essential backbone for the performance of not only the CPU cores, but also the whole rest of the system. In this case we’re taking a closer look at how the memory subsystem behaves on the CPU side of things.

Last year we saw Apple make significant changes to the SoC’s memory subsystem with the inclusion of a new architecture system level cache (SLC), which serves as the last level cache for not only the CPU, but also a lot of other SoC components like the GPU.

Looking first at the linear graphed memory latencies, we see that the A13’s structural DRAM latency falls in at ~104ns, a very slight regression to the 102.8ns of the A12. Apple in this regard isn’t the best among mobile SoCs, HiSilicon’s newest Kirin 990 now falls in at ~96ns and the Exynos 9820 should also fall into a similar range, however this doesn’t matter too much in the grand scheme of things given Apple’s massive cache hierarchy. Patterns such as full random pointer chasing is significantly more performant on Apple’s cores and this should be tightly linked to the strong TLBs as well as iOS’s system configuration choice of using 16KB pages.

Moving to a logarithmic chart we better see the transitions between the cache hierarchies. We can clearly see Apple’s 128KB L1D cache here. The L2 cache is also relatively straightforward till 4MB as the latencies are relatively flat. From here on things become quite complicated and things differ a bit compared to the A12. Last year we determined that the L2 cache structure physically must be around 8MB in size, however we saw that it only looks as if the big cores only have access to around 6MB. Apple employs an “L2E” cache – this is seemingly a region of the big core L2 cache that serves as an L3 to the smaller efficiency cores (which themselves have their own shared L2 underneath in their CPU group).

In this region the new A13 behaves slightly different as there’s an additional “step” in the latency ladder till about 6MB. Frankly I don’t have any proper explanation as to what the microarchitecture is doing here till the 8MB mark. It does look however that the physical structure has remained at 8MB.

Going further out into the cache hierarchy we’re hitting the SLC, which would act as an L3 to the large performance cores, but should be shared with other IP blocks in the SoC. Here we see a significant change in behavior to the A12. If one had to guess as to what happening you’d think that the SLC has grown in size beyond the 8MB we estimated to have been used in the A12. Short of analyzing the die shot and see if the structure indeed has doubled, I’m a bit skeptical and I feel it’s more likely that Apple is using a partitioning system and has possibly enabled the CPU complex to access more of the SLC. What is evident here, is the doubling of the SLC cache from 8MB to 16MB.

We mentioned that the Lightning cores L2 is faster now: Converting the measured latencies from nanoseconds to core cycles, we see the structural speed changes to the caches. The L1 remains at 3 cycles which is massively impressive given its 128KB size. The L2 cache has been reduced from 16 cycles down to 14 cycles, which is again extremely impressive given its purported 8MB physical size. Accounting for the core’s frequency increase, we do more noticeably see that the structural memory latency has increased a bit on the A13, adding about 21-22 cycles. It’s possible that the microarchitectural changes that made the SLC so much faster this generation had a knock-on effect in adding more total latency to DRAM.

Looking at the new Thunder cores versus last year’s Tempest microarchitecture, we see the new cache hierarchy differences. The L1D has grown from 32KB to 48KB – straightforward until now.

The L2 cache size also has evidently increased. Last year we estimated that the small core cluster had 2MB of shared L2, but was partitioned in such as way that a given core only has access to about 1.5MB, and this depth depended on the power management policy and DVFS state of the cores, appearing to only have access to 512KB when at the lowest performance states.

This year, this 1.5MB access size has seemingly increased to 2.5MB. I thus estimate the shared L2 of the small cores has increased from a physical 2MB to 3MB. Past this we’re seeing a step-wise behavior in latency up to 4MB – it’s possible this would be part of the L2E cache of the CPU complex, so in other words we’d possibly be accessing a special partition of the big core’s L2.

Update October 27th: The die shot reveals that the L2 of the Thunder cluster is half the size of the Lightning cluster L2, thus we estimate it's 4MB large in total.

In this graph we continue to see the behavior change of the A13’s SLC. At first glance it appears bigger, which still can be the case, but I rather think the CPU complex has much better access to the 4 (or more) cache slices of the SLC in this generation.

Another change of the new Thunder cores here is that we’re obviously seeing an increase in the L2 TLB capacity of the core. While the L1 TLB seems to have remained unchanged at 128 pages / 2MB, the L2 TLB has increased from 512 pages to 1024 pages – covering up to 16MB, a quite ideal size as it’s covering the depth of the SLC.

Finally, we see that the efficiency cores in the A13 this time around don’t have access to faster DRAM on their own – the memory controller remains very slow and DRAM latencies are in excess of 340ns while on the A12 the Tempest cores were able to enjoy latencies of 140-150ns. This explains some of the performance regressions of the new Thunder cores we measured earlier.

Bandwidth between the A13 and A12 doesn’t majorly differ in the L1 and DRAM regions beyond minor clock speed changes. In the L2, we notice there’s been a more noticeable increase in performance for read+writes into the same cache line, increasing performance by 25%.

It’s again in the SLC region where we see major changes – while on the A12 the bandwidth here slowly fell off in depth, the A13 is able to sustain the same bandwidth over the 16MB of system cache. It’s impressive that the bandwidth here is essentially equal to the L2 – albeit of course quite notably worse latency as we noted earlier. The smaller dips at the 8MB region is an artifact of the cache behavior between the big L2 and the SLC.

Finally, the MLP graphs showcase the memory level parallelism capacity of the CPU cores and the memory subsystem. MLP is the ability for the CPU to “park” memory requests missing the caches and to continue executing in out-of-order fashion other requests. High MLP ability is extremely important to be able to extract the most from out-of-order execution of code, which has higher memory pressure and more complex memory access patterns.

The A13 here again remains quite unique in its behavior, which is vastly more complex that what we see in any other microarchitecture. The non-linearity of the MLP speedup versus access count is something I can’t find a viable explanation for. We do see that the new A13 is a little bit better and more “even” than the A12, although what this practically means is something only Apple’s architects know. In general, Apple’s MLP ability is only second to AMD’s Zen processors, and clearly trounces anything else in the mobile space.

The overall conclusion for the A13’s memory subsystem is that Apple has evidently made very large changes to the system level cache, which is now significantly faster than what we’ve seen in the A12. The L2 cache of the big cores benefit from a 2-cycle latency reduction, but otherwise remain the same. Finally, the new Thunder efficiency cores have seen large changes with increased L1D, L2 and TLB capacity increases.

The Apple A13 SoC: Lightning & Thunder SPEC2006 Perf: Desktop Levels, New Mobile Power Heights
POST A COMMENT

239 Comments

View All Comments

  • Jon Tseng - Wednesday, October 16, 2019 - link

    Nice! Any additional thoughts on the U1 UWB chip. I guess not much you can do with it yet but to me the possibilities are intriguing... Reply
  • Andrei Frumusanu - Wednesday, October 16, 2019 - link

    I think Apple has more plans with it in the future, but yes right now it doesn't do very much. Reply
  • tipoo - Wednesday, October 16, 2019 - link

    Definitely think it's getting the hardware ready for the AR glasses. Hyper precise location tracking just by putting your phone down on a desk and having the U1 chips communicate. Reply
  • Diogene7 - Wednesday, October 16, 2019 - link

    I am dreaming of that the Apple U1 UWB chip could be used in a not too distant future (2020 / 2021) for precise spatial locasization for at (short) distance wireless charging : by knowing where exactly in space an Apple device is, Apple might be able to dynamically and efficiently focus wireless energy transfer maybe through wireless resonant charging (Airfuel) for an iPhone or through RF charging like Energous / Ossia for recharging Apple Airpods from an iPhone...

    I think I am dreaming, but just hope that Apple is working hard to make wireless power at a short distance a reality : I would dream to be able to drop my iPhone anywhere on my bedside table, and that it automatically recharge during the night from a base station up to a distance of 1,5 foot / 50cm : it would bring sooooo much more convenience than Qi wireless charging...
    Reply
  • patel21 - Wednesday, October 16, 2019 - link

    Man, you are lazy. Reply
  • Diogene7 - Wednesday, October 16, 2019 - link

    @patel21 : How many times do you still plug an Ethernet cable to your laptop to surf on internet instead of using WIFI ? WIFI is simply more convenient...

    Similarly, wireless charging at a distance (up to ~ 1,5 foot (50cm)) would be so much more convenient than to have to plug a cable to recharge a device

    It also true for Internet of Things (IoT) devices : tjere seems to be some studies showing that consumers stop using many IoT devices that work on batteries because they have to change the batteries

    I strongly believe that wireless charging at a short distance is a requirement for the sale of IoT sensors to really take off because managing 10s or 100s or more of IoT devices with batteries is not really manageable by consumers in the long run...
    Reply
  • Molbork - Tuesday, June 16, 2020 - link

    And you just halved the power efficiency of your laptop and devices. EM transmission power is 1/r^2, checking your laptop could cost you 2-3x more than a direct connection at longer distances. Reply
  • Henk Poley - Friday, October 18, 2019 - link

    I wonder if they'll do things like heart- and breathing-rate measurement, and counting of people around you (how many hearts). Such as was demonstrated for radar based baby monitoring. Fairly low power, a 'cigarette pack' size device attached to a baby cot could work for half a year by only periodically measuring.

    Could be interesting for meetings, that your phone knows everyone has arrived, people were agitated, etc.
    Reply
  • Adonisds - Wednesday, October 16, 2019 - link

    Why is it required less than double the power to produce twice the display brightness? Reply
  • michael2k - Wednesday, October 16, 2019 - link

    Displays aren't actually perfectly transparent, and the light generating devices might absorb some of the energy instead of transmitting it.

    Increasing transparency is one way to produce more brightness with less energy.
    Reducing the amount of energy absorbed by the LEDs (and thus transformed into heat) is another way to produce more brightness with less energy.
    Changing the LEDs basic chemistry to more efficiently transform electricity into light is a third way.

    Fundamentally less waste heat, more light.
    Reply

Log in

Don't have an account? Sign up now