I feel like somewhat of a geek for saying this but the way I understand it (not very well

), the speed factor is probably related to the kernel issue:
Linux is monolithic-kernel based system. This means most essential services are compiled together all at the same time. This means that it's fast, but it takes work to modify the OS.
Mac OS X is a microkernel-based system. This means that the kernel contains an absolute bare minimum of the services required (memory, disk, process management), and the rest are separate modules sitting on top of the microkernel. This means it's not as fast as a monolithic kernel (since in a microkernel, different parts of the operating system share different memory spaces), but it's easier to reconfigure (since the modules are independent of the kernel itself).
Basically, it's performance vs. configurability. But this is all just my speculation, since I haven't any real experience with either operating system...
Feedback appreciated,
Krishen