Mac OS X versus Linux

Lmbench 2.04 provides a suite of micro benchmarks that measure the bottlenecks at the Unix operating system and CPU level. This makes it very suitable for testing the theory that Mac OS X might be the culprit for the terrible server performance of the Apple platform.

Signals allow processes (and thus threads) to interrupt other processes. In a database system such as MySQL 4.x where so many processes/threads (60 in our MySQL screenshot) and many accesses to the kernel must be managed, signal handling is a critical performance factor.

Larry McVoy (SGI) and Carl Staelin (HP):
" Lmbench measure both signal installation and signal dispatching in two separate loops, within the context of one process. It measures signal handling by installing a signal handler and then repeatedly sending itself the signal."
Host OS Mhz null null
call
open
I/O
stat slct
clos
sig
TCP
sig
inst
Xeon 3.06 GHz Linux 2.4 3056 0.42 0.63 4.47 5.58 18.2 0.68 2.33
G5 2.7 GHz Darwin 8.1 2700 1.13 1.91 4.64 8.60 21.9 1.67 6.20
Xeon 3.6 GHz Linux 2.6 3585 0.19 0.25 2.30 2.88 9.00 0.28 2.70
Opteron 850 Linux 2.6 2404 0.08 0.17 2.11 2.69 12.4 0.17 1.14

All numbers are expressed in microseconds, lower is thus better. First of all, you can see that kernel 2.6 is in most cases a lot more efficient. Secondly, although this is not the most accurate benchmark, the message is clear: the foundation of Mac OS X server, Darwin handles the signals the slowest. In some cases, Darwin is even several times slower.

As we increase the level of concurrency in our database test, many threads must be created. The Unix process/thread creation is called "forking" as a copy of the calling process is made.

lmbench "fork" measures simple process creation by creating a process and immediately exiting the child process. The parent process waits for the child process to exit. The benchmark is intended to measure the overhead for creating a new thread of control, so it includes the fork and the exit time.

lmbench "exec" measures the time to create a completely new process, while " sh" measures to start a new process and run a little program via /bin/ sh (complicated new process creation).

Host OS Mhz fork
hndl
exec
proc
Sh
proc
Xeon 3.06 GHz Linux 3056 163 544 3021
G5 2.7 GHz Darwin 2700 659 2308 4960
Xeon 3.6 GHz Linux 3585 158 467 2688
Opteron 850 Linux 2404 125 471 2393

Mac OS X is incredibly slow, between 2 and 5(!) times slower, in creating new threads, as it doesn't use kernel threads, and has to go through extra layers (wrappers). No need to continue our search: the G5 might not be the fastest integer CPU on earth - its database performance is completely crippled by an asthmatic operating system that needs up to 5 times more time to handle and create threads.

Mac OS X: beautiful but… Workstation, yes; Server, no.
POST A COMMENT

112 Comments

View All Comments

  • iljitsch - Saturday, June 04, 2005 - link

    There seems to be considerable confusion between threads and processes in the review. I have no trouble believing that MacOS doesn't do so well with process gymnastics, but considering the way Apple itself leverages threads, I would assume those perform much better.

    I don't understand why Apache 1.3 was used here, Apache 2.0 has much better multiprocessor capabilities and would have allowed to test the difference between the request-are-handled-in-processes and requests-are-handled-in-threads ways of doing things.
    Reply
  • Phil - Saturday, June 04, 2005 - link

    #79: Wow. I had no idea that they were actually going to do it, I had assumed that it was typical industry nonsense!
    If this is true, then IMHO Apple won't be in much of a better position (with regards to this article) as they'll still need to work on the OS, regardless.

    Can anyone speculate as to why they *really* want to switch to x86/Intel? I wonder if they'll consider AMD too...
    Reply
  • rorsten - Saturday, June 04, 2005 - link

    Uhm, the estimation for power consumption is completely wrong. The only significant CMOS power consumption - especially for an SOI chip - is the current required to charge or discharge the gates of the FETs, which only happens when a value changes (the clock accounts for most of the power consumption on a modern synchronous chip). Since we're talking about current only, this is purely resistive power, I^2R style, and since the current is related to the number of transitions per second, increasing the clock rate linearly increases the current which quadratically increases the power consumption. Reply
  • kamper - Saturday, June 04, 2005 - link

    Here's another story about Apple and Intel from cnet:
    http://news.com.com/Apple+to+ditch+IBM%2C+switch+t...

    Interesting in the context of this article but I won't believe it without much more substantial proof :)

    +1 on getting a db test using the same os on all architectures whether it be linux or bsd

    +1 on fixing the table so that it renders in firefox
    Reply
  • shanep - Saturday, June 04, 2005 - link

    Re: NetBSD.

    Sorry, I just noticed it is not supported yet by NetBSD.

    Forget I mentioned it.
    Reply
  • shanep - Saturday, June 04, 2005 - link

    "Wessonality: Our next project if we can keep the G5 long enough in the labs."

    How about testing these machines with NetBSD 2.0.2 to keep the hardware comparison on as close an equal footing as possible.

    This should mostly remove many red herrings associated with multiple differences in software across different hardware.
    Reply
  • michaelok - Saturday, June 04, 2005 - link

    "i've had for awhile about OS X server handling large numbers of thread. My OS X servers ALWAYS tank hard with lots of open sessions, so i keep them around only for emergencies. T"

    Moshe Bar (openMosix) has been an avid Mac follower for years, I see he has a few suggestions for OSX, including ditching the Mach so you can run FreeBSD natively, which has much better peformance. In fact, thread performance is one of FreeBSDs strong points, although Linux has largely caught up.

    Also research his Byte articles, you can see how a proper comparison can be done, although he does not claim to be a benchmarking expert.

    http://www.moshebar.com/tech/?q=node/5
    http://www.byte.com/documents/s=7865/byt1064782374...
    Reply
  • johannesrexx - Saturday, June 04, 2005 - link

    Everybody should use Firefox by default because it's far more secure. Use IE only when you must. Reply
  • michaelok - Saturday, June 04, 2005 - link

    "with one benchmark showing that the PowerMac is just a mediocre PC while another shows it off as a supercomputer, the unchallenged king of the personal computer world."

    Well, things are a little different when you connect, say, 32768 processors together, i.e. you go from running MySQL to Teradata, so yes, the Power architecture seems to dominate, and the Virginia Tech supercomputer is still up there, at 7th.

    http://www.top500.org/lists/plists.php?Y=2004&...

    " The RISC ISA, which is quite complex and can hardly be called "Reduced" (The R of RISC), provides 32 architectural registers"

    'Reduced Instruction Set' is misleading, it actually refers to a design philosophy of using *smaller, simpler* instructions, instead of a single complex instruction. This is to be compared with the Itanium for example, which Intel calls 'EPIC' (Explicit Parallel Instruction Computing), but it is essentially derived from VLIW (Very Long Instruction Word).

    Anyway, nice article, certainly much more to discuss here, such as SMT (Simultaneous Multithreading), (when that is available for the Apple :), vs. Intel's Hyperthreading. We'll still be comparing Apples to Oranges but isn't that why everybody buys the Motor Trend articles, i.e. '68 Mustang vs. '68 GT?


    Reply
  • psychodad - Saturday, June 04, 2005 - link

    I agree. Recently I read a review which pitted macs against pcs using software blatantly optimized for macs. If you have ever used unoptimized software, you will know it. It is slow, often unstable and not at all usable, especially if you're after productivity. Reply

Log in

Don't have an account? Sign up now