Linux Scalability Project

Status report for November and December 1998

Center for Information Technology Integration
School of Information at the University of Michigan

The primary goal of this research is to improve the scalability and robustness of the Linux operating system to support greater network server workloads more reliably. We are specifically interested in single-system scalability, performance, and reliability of network server infrastructure products running on Linux, such as LDAP directory servers, IMAP electronic mail servers, and web servers, among others.

Summary

The major focus of the first few months has been to create structure and scope for the project, and to begin tackling the learning curve. We've begun building our connections to Netscape staff and to major Linux developers. Stage one of the project workscope has begun.

Milestones

Challenges

Netscape's server developers have spent considerable effort analyzing and architecting a threading model for network servers. This threading model works well under Windows NT, but none of the Unix-based platforms support it efficiently. Our developers are interested in having Linux support Windows NT-style thread dispatching and asynchronous/queued I/O programming interfaces because they are clean and are known to perform well. Linux builds on POSIX standards, which rely heavily on signals and POSIX threads, and are not directly compatible with Windows NT programming constructs. There is fear that the Linux model won't perform well, and will be difficult to use because of the unknowns involved in mixing signals and threads. Linux kernel developers want to stay with standard interfaces, but have agreed to consider a "shared signal queue" model to help support Netscape's architecture of queued I/O and event-driven thread dispatching. A shared signal queue may or may not meet the performance and programming ease requirements set by server developers.

It is important for us to understand that the forces driving the evolution of Linux are different than those driving commercially-developed operating systems. While market forces influence a commercial operating system's feature set, the Linux feature set is determined by what the developers believe is right, based on their desires to have a high-performance standards-based operating system. Since they are not paid for their work, they are not attracted by calls for support of features that may increase marketability, or that may provide support for other commercial software. In other words, they do not work on Linux for customers, but for themselves, for aesthetic reasons. Therefore, if CITI and Netscape wish to influence the design and implementation of Linux, we must get involved with their process, in order to prove the value of our involvement because we have good ideas, and because we have faith in, and respect for, open source. Our involvement needs to take the form of providing source code, patches, and especially detailed benchmark results that back up our good ideas and demonstrate our value to Linux.

Towards that end, the Linux Scalability Project can provide some focused effort addressing specific performance and scalability problems in Linux. In addition to this effort, Netscape can also advance its enterprise Linux agenda by continuing to provide a wide range of support for enterprise Linux. We have begun to make meaningful contributions to the Linux source stream, but there is a high hurdle to surmount before our contributions are fully accepted into Linux.

Performance graphs

If you have comments or suggestions, email linux-scalability @ citi.umich.edu

Copyright 1998