An excerpt from "Inside Windows NT" by Helen Custer
Foreword by David N. Cutler
In 1965, I graduated from college with a B.A. in mathematics, a minor in physics, and an overwhelming desire to be an engineer and to build things. So I took a job with DuPont in Wilmington, Delaware, as a materials testing engineer. After about a year of absolute boredom, I was lent to the mathematics and statistics group assigned to construct a computer simulation model for a new foam-making process that the Scott Paper Company was developing. Working with machines that never did what I meant them to was humiliating, but within six months I was hooked, and what I had shunned coming out of school -- computers -- turned into my life's vocation.
Soon after, I transferred to DuPont's engineering department, where I could program full time. DuPont had a small group that built online computer system applications. My real motivation for joining this group was to get closer to computers, and in fact, I wanted to work on implementing an operating system. While in the group, I had the good fortune to work on several stand-alone real-time systems where the project involved writing the central control program that scheduled the various tasks and monitored system activity as well as writing the actual application code.
It soon became apparent that the only way I was going to get the opportunity to work on implementing a real operating system was to join a company that made computers for a business. And so in 1971 I left DuPont for a job in Maynard, Massachusetts, with Digital Equipment Corporation (DEC). As it turned out, this put me in the operating system business for quite some time to come. Little did I know that I would be fortunate enough to develop several operating systems in my lifetime; developing one is a rare opportunity for anyone.
My first operating system project was to build a real-time system called RSX-11M that ran on Digital's PDP-11 16-bit series of minicomputers. At the time, our goals seemed very ambitious. We were asked to build a multitasking operating system that would run in 32 KB of memory with a hierarchical file system, application swapping, real-time scheduling, and a set of development utilities. The operating system and utilities were to run on the entire line of PDP-11 platforms, from the very small systems up through the PDP-11/70 which had memory-mapping hardware and supported up to 4 MB of memory.
I have many fond memories of how RSX-11M took shape, I had a rubber stamp made that proclaimed "Size Is the Goal" and proceeded to stamp every last bit of correspondence to make sure that all the programmers and project managers understood how important it was to achieve our goals. We also learned the power of conditional assembly (high level language use in operating systems was in its infancy at this time), and whenever someone added a feature, we just made it a system-generation option.
While developing RSX-11M, we spent most of our time engineering solutions to memory problems. Because the system had to run in 32 KB, we generated a memory budget that divided available memory equally between the operating system and the utility programs. That left a mere 16 KB for utility programs and led to long hours tuning overlay structures to achieve acceptable performance for many of the RSX-11M system programs.
Although RSX-11M had some very stringent size and performance constraints, of the systems I've worked on it was probably the easiest one to develop. It involved re-implementing an existing system but allowed us the freedom to change and subset the programming interfaces as long as programs could be reassembled or recompiled with minimal source-code changes. RSX-11M was introduced in 1973, 18 months after we started building it. It proved to be very successful and helped make the PDP-11 the most popular 16-bit minicomputer of its time.
The PDP-11 provide better price/performance than mainframes, was affordable at the department level, and along with other popular minicomputers of the same era, led to the first wave of "downsizing" in the computer industry. Downsizing was an attempt to "bring down" mainframe applications to the minicomputer systems. Many of the mainframe programs were larger than the PDP-11 could easily accommodate, and almost immediately Digital was up what Gordon Bell has deemed the single most important reason that computer architectures become obsolete: the lack of enough address bits.
Out of this need, the VAX architecture was born, and it became one of the most popular architectures of the late '70s and remained popular throughout the '80s. The VAX architecture provided 32 bits of virtual address space and eliminated the need to wrestle programs into what seemed to be an ever-decreasing amount of virtual address space.
My second opportunity to develop an operating system arrived with the VAX. I was very fortunate to be chosen to lead the operating system effort for the VAX-11 architecture, the result of which was the VMS operating system.
VMS was Digital's second general-purpose time-sharing system, developed specifically for the VAX architecture. Because the VAX architecture had grown out of the tremendous success of the PDP-11, however, this time it was mandatory to provide more than source-level compatibility for applications.
Thus the VAX-11 architecture included a PDP-11 compatibility mode which PDP-11 instructions were executed directly by hardware. At that time, it was inconceivable that a single operating system could support more than "compatibility environment". Although it wasn't the best known of the PDP-11 operating systems (amazingly, Digital had no fewer the 10 PDP-11 operating systems at one time or another!), RSX-11M was chosen as the operating system interface that would be emulated in PDP-11 compatibility mode on the VAX. This decision probably didn't make sense to a number of people outside the company, but RSX-11M had the largest number of application development tools, and had the most general-purpose operating system features, supporting multitasking, and had a file system structure that could be compatibly extended. Ultimately, the VAX-11 systems ran the RSX-11M binaries right off the distribution kit; it allowed RSX-11M volumes to be directly mounted and their files to be accessed and shared between RSX-11M compatibility-mode programs and native VMS programs.
From a technical perspective, the biggest mistake we made in VMS was not writing it in a high level language. At that time, we had a group of very accomplished assembly language programmers, some stringent size constraints, and no complier with the appropriate quality for operating system development. So to ensure that we would ship the system in a marketable time frame, we wrote it in assembly language. Looking back on what happened, it would still be hard to make the decision to write VMS in a high-level language. (Moral: The right thing to do technically isn't always the best thing to do financially.)
Early in the '80s, while minicomputers were busy absorbing mainframe and other new applications, two important technologies were emerging: the personal computer (PC) and workstations. After the VMS project, I spent a few years developing compliers and then led a group that built Digital's first MicroVAX workstation -- the MicroVAX I.
Workstations like the MicroVAX provided individual, high-performance computing for applications such as computer-aided design (CAD), whereas PCs supported business applications aimed at personal productivity, such as spread sheets and word processors -- two very successful early PC products. Although workstations were relatively pricey, personal computers had to be affordable to small businesses.
In order to meet price objectives, the original PCs were built with 8-bit, and later with 16-bit, microprocessors. They were constrained in much the same way RSX-11M had been and required considerable effort on the part of programmers and operating system designers to accommodate their limitations. Hardware resources were so scarce that operating systems existed mainly to handle a few low-level hardware functions and to provide a set of file system libraries. But the personal computer offered something that minicomputers did not -- a market in which independent software developers could sell their programs at a higher volume. As a result, the breadth and variety of applications that run on PCs and exploit their capabilities is truly amazing.
In the mid-'80s, microprocessors gained 32-bit addressing, and workstations were quick to take advantage of this capability. However, because of the very large installed base of personal computers and their applications, it was not easy to simply roll in another computer and then recompile and relink all the application software. End users of PCs simply didn't have the source code for all their programs, and they demanded binary compatibility.
In the summer of 1988, I received and interesting call from Bill Gates at Microsoft. He asked whether I'd like to come over and talk about building a new operating system at Microsoft for personal computers. At the time, I wasn't too interested in working on personal computers, but I thought this would be a good time to meet Bill and discuss what he had in mind. What Bill had to offer was the opportunity to build another operating system, one that was portable and addressed some of the concerns people had about using personal computers to run mission critical applications. For me, it meant the chance to build another operating system!
Bill finally convinced me that this was an opportunity I couldn't pass up, and in October of 1988, I came to Microsoft and started to build the team that would build the new operating system. I didn't realize it at the time, but this would be the most ambitious operating system project on which I had ever embarked.
Our goals for the system included portability, security, POSIX compliance, compatibility, scalable performance (multi processor support), extensibility, and the ease of internationalization. Of all these goals, by far the one that was hardest to achieve and that had the most profound effect in the structure of the system was compatibility. Hundreds of thousands of PDP-11 systems had been sold, but tens of millions of personal computers were in operation! As if that weren't enough, we needed to compatibly support three separate 16-bit operating environments and add new 32-bit capabilities to free personal computer applications from the same kind of virtual address constraints that had existed for the PDP-11. To top it off, we wanted to support the UNIX standard interface specification called POSIX.
Now almost four years later, we are on the brink of bringing this system, Windows-NT, to market. Helen Custer started work on this book when the operating system design began. As our design has matured, the book has undergone continual change to track the operating system architecture. This has been an arduous task -- keeping up-to-date and writing and rewriting the various chapters of the book as the design evolved. Although it is our design, Helen is the one who has captured the essence of that design and made it understandable to more than just serious operating system implementers. For this, we owe Helen a great debt.
It is impossible to acknowledge all the people who have contributed to the design of Windows NT. I must say that I did not design Windows NT -- I was merely one of the contributors to design the system. As you read this book, you will be introduced to some, but not all, of the other contributors. This has been a team effort and has involved several hundred-person years of effort. Perhaps the most important contribution of all was made by the people who have tested and stressed the system. Without their effort, Windows NT could not have achieved the level of quality that it has achieved.
I hope you enjoy this book about Windows NT as much as we enjoyed designing the system.
Director, Windows NT Development
Copyright (c) 1993 by Microsoft Press