Linux Scalability Project

Status report for September and October, 1999

Center for Information Technology Integration
School of Information at the University of Michigan

The primary goal of this research is to improve the scalability and robustness of the Linux operating system to support greater network server workloads more reliably. We are specifically interested in single-system scalability, performance, and reliability of network server infrastructure products running on Linux, such as LDAP directory servers, IMAP electronic mail servers, and web servers, among others.


We're becoming more familiar with the Linux development process, and Linux performance and scalability issues. We're continuing to reach out to potential sponsors. Work continues on long-term projects.



Linux Development Road Map

The actively maintained Linux kernel releases always arrive in pairs: an even-numbered "stable" kernel release which is meant for production systems and reliable desktop applications; and an odd-numbered "development" kernel release, which has new features and major overhauls underway. The development kernel is supposed to be more advanced than its counterpart stable kernel, and will one day become the next stable release.

Today, a Linux development kernel becomes a stable release in an almost arbitrary way. Linus decides how long the release development cycle will take, and then adds new features until he's ready to freeze the code base for the stable release. Then, after a stable release arrives, we all look over our shoulders and ask ourselves what we accomplished.

In fact, the "stable" release isn't really stable at all, since features and new drivers continue to be added. One of the 2.2 kernel's biggest problems is that many of the features which were added in the 2.1 kernel were not finished when 2.2.0 arrived. Specific examples of this phenomena include:

Commercial enterprises want to know when they can expect certain features to appear and become stable in any kernel. Kernel developers, however, have been arbitrarily adding such features as large memory support and application level access to raw block I/O. It would be better to provide some codified mechanism for determining if/when features will get into the kernel, in a way that preserves both kernel developer sanity and kernel release integrity.

One thing that I would like to see integrated into the Linux kernel development process, either socially or technologically, is a somewhat formal long-term feature development plan. A development plan, and especially a process for reaching consensus about it, would help the community decide what features should be added in a development cycle, and what qualifies each feature as ready for a stable release (that is, what tests the kernel has to pass before an odd numbered release becomes an even numbered one). Especially because Linux kernel development is a distributed process, it would be useful to have shared information about:

The idea of developing a list of release requirements by building a consensus, then publicly announcing those requirements, may not appeal to some members of the development community. After all, developing the Linux kernel has always been a hobby, not a professional obligation; having a system by which to set goals and determine progress feels like Big Brother. For many, comparing progress to a list of goals is a mixed bag of "shoulds," "could haves," and "we made its". Perhaps volunteer developers may feel that this should be the job of the commercial Linux distributors, as is quality assurance and performance testing. I believe that, like QA and performance work, release integrity and feature plans must be part of the process from its first steps.

The "cathedral"-style approach described above may conflict with the Linux community's devotion to "bazaar" development processes. In order to make a development planning process work well, these anxieties must be met with positive and creative solutions, not glib party-line answers. There is no good reason that such a planning process has to add to the stress that already plagues kernel developers. Hopefully, a good planning process can ease developers' Linux-related work load by adding a useful channel for communication and decision-making.

If you have comments or suggestions, email linux-scalability @

Copyright 1999