Mac OS X Internals

Mac OS X took the unassimilated, thinking and computer-literate world by storm since its release on the 24th of March 2001. Its combination of commercial, high quality software applications, a state-of-the-art, ever-evolving and well thought out desktop environment and the solid Unix underpinnings that came with NeXT’s acquisition, gave it a significant part of its ‘holy-grail’ appeal that has so far eluded both the Classic Mac OS, Microsoft’s clone thereof and more recent efforts for desktop evolution on linux.
Yet, besides the æsthetics and usability aspects as well as the appeal that OS X may have on the general population, it is a technically very interesting engineering and `cultural’ feat, as it represents the amalgamation of a number of previously seemingly orthogonal (or incompatible if you prefer) technologies, ironically offering a largely complementary set of features: the stability, security and familiarity of FreeBSD, the object oriented driver model of NeXT, the once-promising-often-dismissed microkernel Mach, the legendary ease of use and intuitiveness of the Macintosh. While most people will probably feel right at home and include OS X in their daily computer use routine, without bothering to spend time or money on any of the introductory books directed at the casual user of the OS, there is a significant (and, I feel, ever-increasing) part of the OS X userbase that comes from the adventurous Unix world and is keen on learning the intricate details of the system. It is for this group that Mac OS X Internals was written for. And with more than 1600 pages in 12 chapters, it certainly contains a wealth of information for the curious or technically proficient user.

The book starts with a historical review of Mac OS and its predecessors at Apple which, albeit short, seems thematically complete. While not immediately useful, the text is easy to read and provides some background on technologies that still live on in one form or another in OS X. It continues with an technical overview of OS X, a preamble for the chapters that follow: the overview covers everything from the Open Firmware (found on older PowerPC Macs), to the BootX bootloader, Darwin and the xnu kernel, the file system, Mach-O and the runtime, the OS X C library, the concepts and implementations of bundles and frameworks (which are significantly different to their counterparts on Windows or other Unix systems), Core and Application Services, the Application Environments, including BSD, X11, Cocoa and Carbon, Java, Classic, Rosetta, QuickTime and WebObjects. It briefly touches upon the User Interface and specifically the effects, the resolution-independent features that started creeping in since version 10.4 and will probably appear in 10.5 ‘Leopard’, Universal Access. It continues with an overview of programming tools and utilities for OS X, namely XCode, gcc etc., Security and System Administration, both at the Kernel and the User spaces, Networking (which is largely industry standard) and OS X Server features such as XGrid and Xsan.
Beyond the overview the book spends a considerable amount of ink and paper on the architecture of the PowerPC 970 and 970FX, aka. the G5 (about 100 pages) which while interesting and useful today will probably seem very outdated in a couple of years’ time. It is also quite detailed on its presentation of OpenFirmware, the firmware used in all ‘new world’ PowerPC Macs. Again while interesting OpenFirmware was abandoned in favour of Intel’s EFI. And while EFI is discussed for 23 pages, OpenFirmware gets most of the remainder of that chapter (about 90 pages).
Fortunately, these first few chapters is where the expiring material ends. The book continues with detailed discussions of the kernel and user-level startup, the xnu kernel itself, processes and memory, IPC (where one may find a very useful presentation of the various IPC layers found on OS X), driver development and the I/O kit, file systems, including a presentation of Spotlight and a detailed presentation of the HFS+ filesystem, the default and most common filesystem used by OS X. This is where the book was probably supposed to end were it not for the switch to Intel. Assuming the lack of time and advanced stage of the writing process, the author has added an Appendix of about 12 pages were differences and characteristics of the Intel Mac computers are discussed. Specifically, the Appendix presents a brief overview of the firmware, hardware, byte ordering changes, Rosetta and a list of kernel differences, such as thread creation, system calls etc.

New platform, New Kernel, New Rules

It has been rumoured for a long time that Leopard might bring considerable changes to the kernel as the current hybrid’s performance is significantly lower to that of other modern kernels, e.g. Linux. One possibility mentioned by John Siracusa on Ars Technica [1][2], was the replacement of the xnu by another kernel, something technically possible within the timeframe of Leopard’s release and increasingly probable after the departure of Avie Tevanian earlier this year. One seemingly good candidate that has attracted the attention of the technically-savvy Mac community is L4, the modern microkernel successor to Mach that largely diminishes the performance gap between it and monolithic kernels such as Linux. However, no sign of L4 (or another kernel for that matter) has appeared in the β versions of OS X that have been distributed by Apple to its ADC members. xnu seems to still be there, with no mention of a new kernel by Apple on the related web page. Nevertheless, Jobs talked of more features in Leopard than those presented in WWDC back in August and improvements on the kernel are more or less certain. The extend to which those improvements will render the relevant content of this book obsolete is, however, unknown.

Final words

I found the book thorough and extremely valuable, especially for those developing large applications or system software on OS X. While most of the content of the book can be found online, in reference manuals by Apple and IBM, the value lies with the presentation and the fact that so much information is collected in one volume and neatly arranged and indexed. This is definitely not an introductory text, and some experience with the C programming language and Operating System design and implementation is essential to appreciating its content. Most Computer Science graduates (or students) should be comfortable with it, but if you have no experience with Operating System internals you should probably read an introductory text first. While OS X is bound to evolve much faster than Linux or Windows in the coming years, much of the information contained in this book is likely to be of some relevance for years to come.
For those interested, the book’s author, Amit Singh, is maintaining two web sites relevant to technology: his personal site, kernelthread and a site for the book, where you can find errata, updates, and some new material.