The Mac Pro Review (Late 2013)
by Anand Lal Shimpi on December 31, 2013 3:18 PM ESTThe PCIe Layout
Ask anyone at Apple why they need Ivy Bridge EP vs. a conventional desktop Haswell for the Mac Pro and you’ll get two responses: core count and PCIe lanes. The first one is obvious. Haswell tops out at 4 cores today. Even though each of those cores is faster than what you get with an Ivy Bridge EP, for applications that can spawn more than 4 CPU intensive threads you’re better off taking the IPC/single threaded hit and going with an older architecture that supports more cores. The second point is a connectivity argument.
Here’s what a conventional desktop Haswell platform looks like in terms of PCIe lanes:
You’ve got a total of 16 PCIe 3.0 lanes that branch off the CPU, and then (at most) another 8 PCIe 2.0 lanes hanging off of the Platform Controller Hub (PCH). In a dual-GPU configuration those 16 PCIe 3.0 lanes are typically divided into an 8 + 8 configuration. The 8 remaining lanes are typically more than enough for networking and extra storage controllers.
Ivy Bridge E/EP on the other hand doubles the total number of PCIe lanes compared to Intel’s standard desktop platform:
Here the CPU has a total of 40 PCIe 3.0 lanes. That’s enough for each GPU in a dual-GPU setup to get a full 16 lanes, and to have another 8 left over for high-bandwidth use. The PCH also has another 8 PCIe 2.0 lanes, just like in the conventional desktop case.
I wanted to figure out how these PCIe lanes were used by the Mac Pro, so I set out to map everything out as best as I could without taking apart the system (alas, Apple tends to frown upon that sort of behavior when it comes to review samples). Here’s what I was able to come up with. Let’s start off of the PCH:
Here each Gigabit Ethernet port gets a dedicated PCIe 2.0 x1 lane, the same goes for the 802.11ac controller. All Mac Pros ship with a PCIe x4 SSD, and those four lanes also come off the PCH. That leaves a single PCIe lane unaccounted for in the Mac Pro. Here we really get to see how much of a mess Intel’s workstation chipset lineup is: the C600/X79 PCH doesn’t natively support USB 3.0. That’s right, it’s nearly 2014 and Intel is shipping a flagship platform without USB 3.0 support. The 8th PCIe lane off of the PCH is used by a Fresco Logic USB 3.0 controller. I believe it’s the FL1100, which is a PCIe 2.0 to 4-port USB 3.0 controller. A single PCIe 2.0 lane offers a maximum of 500MB/s of bandwidth in either direction (1GB/s aggregate), which is enough for the real world max transfer rates over USB 3.0. Do keep this limitation in mind if you’re thinking about populating all four USB 3.0 ports with high-speed storage with the intent of building a low-cost Thunderbolt alternative. You’ll be bound by the performance of a single PCIe 2.0 lane.
That takes care of the PCH, now let’s see what happens off of the CPU:
Of the 40 PCIe 3.0 lanes, 32 are already occupied by the two AMD FirePro GPUs. Having a full x16 interface to the GPUs isn’t really necessary for gaming performance, but if you want to treat each GPU as a first class citizen then this is the way to go. That leaves us with 8 PCIe 3.0 lanes left.
The Mac Pro has a total of six Thunderbolt 2 ports, each pair is driven by a single Thunderbolt 2 controller. Each Thunderbolt 2 controller accepts four PCIe 2.0 lanes as an input and delivers that bandwidth to any Thunderbolt devices downstream. If you do the math you’ll see we have a bit of a problem: 3 TB2 controllers x 4 PCIe 2.0 lanes per controller = 12 PCIe 2.0 lanes, but we only have 8 lanes left to allocate in the system.
I assumed there had to be a PCIe switch sharing the 8 PCIe input lanes among the Thunderbolt 2 controllers, but I needed proof. Our Senior GPU Editor, Ryan Smith, did some digging into the Mac Pro’s enumerated PCIe devices and discovered a very familiar vendor id: 10B5, the id used by PLX Technology. PLX is a well known PCIe bridge/switch manufacturer. The part used in the Mac Pro (PEX 8723) is of course not listed on PLX’s website, but it’s pretty close to another one that PLX is presently shipping: the PEX 8724. The 8724 is a 24-lane PCIe 3.0 switch. It can take 4 or 8 PCIe 3.0 lanes as an input and share that bandwidth among up to 16 (20 in the case of a x4 input) downstream PCIe lanes. Normally that would create a bandwidth bottleneck but remember that Thunderbolt 2 is still based on PCIe 2.0. The switch provides roughly 15GB/s of bandwidth to the CPU and 3 x 5GB/s of bandwidth to the Thunderbolt 2 controllers.
Literally any of the 6 Thunderbolt 2 ports on the back of the Mac Pro will give you access to the 8 remaining PCIe 3.0 lanes living off of the CPU. It’s pretty impressive when you think about it, external access to a high-speed interface located on the CPU die itself.
The part I haven’t quite figured out yet is how Apple handles DisplayPort functionality. All six Thunderbolt 2 ports are capable of outputting to a display, which means that there’s either a path from the FirePro to each Thunderbolt 2 controller or the PEX 8723 switch also handles DisplayPort switching. It doesn’t really matter from an end user perspective as you can plug a monitor into any port and have it work, it’s more of me wanting to know how it all works.
Facebook
Twitter
RSS
267 Comments
View All Comments
wallysb01 - Friday, January 3, 2014 - link
Actually, you can upgrade the CPU. What you can’t do is replace the monitor.Liquidmark - Friday, January 3, 2014 - link
You can attach external monitors to the iMac.Liquidmark - Friday, January 3, 2014 - link
"The problem with saying Apple's pricing isn't out of touch with reality is that you can't only compare this to high end workstation's from other companies..."This is a workstation. It has workstation components and is formally classified as such, so you kinda have to compare it to *gasp* OTHER workstations and match their spec as closely as possible to see if the price of the Mac Pro is reasonable or not. Anand is absolutely correct in comparing this to a HP Z420 which is HP's mid-range workstation right now.
"this is Apple's only desktop-ish device."
Ever heard of the Mac Mini?
"Apple doesn't provide options for people who want a high end notebook or desktop for normal use.."
Ever heard of the Macbook Pro or iMac?
You can't discount the fact that the Mac Pro has Xeons under the hood just because you don't like the other options Apple offers. If the Mac Pro has Xeons under the hood, then you have to factor that into the price of the device. You don't get to ignore the engine in a Bently to claim that a Bently is thus overpriced when compared to a Toyota Corolla. You don't get to say that it should have an engine from a Toyota Corolla and that theuy shouldn't have the luxury features and hand-crafted attention to detail that come with a bently. If you want a Toyota corolla, go buy a Toyota Corolla. If you want a Dodge viper, go buy that. Don't tell Bently to make a Toyota Corolla or a Dodge viper and don't expect to buy a bently at the cost of a toyota corolla or dodge viper either because you seem to dismiss the facts that there are differences between the three.
"it's at least 2x as expensive as it should be for it's base unit"
Not according to actual price comparisons it isn't..
Bobs_Your_Uncle - Thursday, January 2, 2014 - link
I'm still wondering how that Nokia Lumia 1020 review is coming along !?p51mustang6 - Thursday, January 2, 2014 - link
You should really do research, just a little, prior to making a review like this, you make a bold statement saying how the Mac Pro is so great and for so cheap, yet you compared it to two companies far from known for making anything professional. Try comparing the Mac Pro to The Origin Genesis Pro-X2 from Origin PC. It starts out with considerably higher specs with a LOWER price tag. They also offer up to dual Intel Xeon E5-2697 Dodeca-core processors (that's 12 cores each CPU for those of you who couldn't handle that) for a total of 24 cores (or twice that of the Mac Pro), up to dual 12GB NVIDIA Quadro K6000s (Apple doesn't even offer anything even closely comparable lol), up to 256GB of RAM (Apple offers up to 64GB), up to 4TB of SSD storage (compared to Apple's 1TB, granted PCI), comes standard with liquid cooling (Apple does not offer), up to an additional 12GB NVIDIA Tesla K40 (once again Apple offers nothing of the sort), Origin comes with one year warranty upgradable to 3 years but also comes standard with LIFETIME support with 24/7 United States based support (I wonder where Apple's support that you get 90 days of is based...lol) The starting price of the Origin is $3,712 compared to $3,999 of the Mac Pro which does not come with dual processors. The trash can is a complete rip off which requires you to go out and use their thunderbolt ports in order to do any real upgrading so you will have random things sitting on your desk, the Origin perhaps bigger, but at least all the goods will always be inside of it. Instead of spending all their time trying to make a computer a cylinder maybe Apple should have tried to compete with the real heavy hitters such as Origin PC.Louiek - Thursday, January 2, 2014 - link
Hi, I am currently myself trying to compare a maxed out mac pro ~ $10k CAD with other OEM workstations of similar spec. I looked into origin but I can't seem to build a similar spec'd (i.e. single Xeon E5-2697v2 etc) that will cost under $11k CAD. Is there something I am missing as your comment leads me to believe that I can build a cheaper PC with origins with similar specs.Liquidmark - Friday, January 3, 2014 - link
You can't the Origin machine only offers extreeeeme options that are ideal for gaming with neon lights. Its solution to things is to throw more cores at it and throw more ram at it even though the ram is slower...stingerman - Thursday, January 2, 2014 - link
Sorry dude, triedto configure a comparable system and it costs more than the Mac Pro...Liquidmark - Friday, January 3, 2014 - link
Ok, I'll bite...Mac Pro:
2.7GHz 12-core with 30MB of L3 cache
32GB (4x8GB) of 1866MHz DDR3 ECC
256GB PCIe-based flash storage
Dual AMD FirePro D700 GPUs with 6GB of GDDR5 VRAM each
User's Guide (English)
$8000 and weighs roughly 11 pounds
GENESIS Pro-X2
ASUS Z9PE-D8 WS
Dual ORIGIN FROSTBYTE 120 Sealed Liquid Cooling Systems
Dual Intel XEON E5-2630 v2 Hex-Core 2.6GHz (3.1GHz Turbo) 15MB Cache (That's 12 cores at a lower clock than the Mac Pro build)
1000 Watt Corsair RM1000
Dual 6GB NVIDIA Quadro 6000 (Non-SLI)
32GB Kingston ECC 1600MHz (4x8GB)
Genuine MS Windows 7 Professional 64-Bit Edition
250GB Samsung 840 Evo Series
ASUS 24X CD/DVD Burner
On Board Audio
Onboard Network Port
ORIGIN Wooden Crate Armor
1 Year Part Replacement and 45 Day Free Shipping Warranty with Lifetime Labor/24-7 Support
ORIGIN Recovery USB3.0 Flash Drive
ORIGIN PC G8 T-Shirt XL
Microsoft Internet Explorer
$11,017 and weighs over 70 pounds.
Now, before anyone says anything, the tee shirt was free and the water cooling was the only offer plus they give a free games offer that I didn't take. Tho I probably should since apparently workstations are all about pro gaming, neon lights and being extreeeeeme.
Liquidmark - Friday, January 3, 2014 - link
Also, if anyone wants to argue that you can get dual 12-core on the origin machine, I'll simply point out that, at spec, I'd almost be able to buy two 12 core Mac Pros. Just saying.