FreeBSD Monthly Developer Status Report

Path: archiver1.google.com!news2.google.com!news1.google.com!sn-xit-02!supernews.com!feedme.news.mediaways.net!newsfeeds.belnet.be!news.belnet.be!news.tele.dk!small.news.tele.dk!129.240.148.23!uio.no!nntp.uio.no!ifi.uio.no!internet-mailinglist
Newsgroups: fa.freebsd.hackers
Original-Date: Wed, 26 Dec 2001 11:10:09 -0500 (EST)
From: Robert Watson <rwat...@freebsd.org>
To: develop...@freebsd.org, hack...@freebsd.org
Subject: October and November FreeBSD Monthly Developer Status Report
Original-Message-ID: <Pine.NEB.3.96L.1011226110904.39578h-100000@fledge.watson.org>
MIME-Version: 1.0
Content-Type: TEXT/PLAIN; charset=US-ASCII
Sender: owner-freebsd-hack...@freebsd.org
Precedence: bulk
Organization: Internet mailing list
Date: Wed, 26 Dec 2001 16:11:36 GMT
Message-ID: <fa.lkjksrv.rksh2n@ifi.uio.no>
Lines: 547


This report was produced with the assistance of Chris Costello.


--


                          November 2001 Status Report

                                  Introduction

   This months report covers activity during the second half of October, and
   the month of November. During these months, substantial work was performed
   to improve system performance and stability, in particular addressing
   concerns regarding regressions in network performance for the TCP
   protocol, and via the introduction of polled network device driver
   support. Work continues on long-term architectural projects for 5.0,
   including KSEs, NEWCARD, and TrustedBSD, as well as the cleaning up of
   long-standing problems in FreeBSD, such as PAM integration. Administrative
   changes are also documented, including work to redefine and formalize the
   release engineering process, and the approval of a new portmgr group which
   will administer the ports collection.

   FreeBSD users and developers are strongly encouraged to attend the USENIX
   BSD Conference in February of next year; it is expected that this will be
   a useful forum both for learning about FreeBSD and on-going work, as well
   as providing an opportunity for developers to work more closely and act as
   a vehicle for discussion and round-the-clock hacking. More information is
   available at the USENIX web site.

   Robert Watson

     * ATA Project Status Report
     * Device Polling
     * Fibre Channel Support
     * FreeBSD 4.5 Release Engineering
     * FreeBSD C99 & POSIX Conformance Project
     * FreeBSD in Bulgarian
     * FreeBSD NVIDIA Driver Port
     * GEOM - generalized block storage manipulation
     * Improving FreeBSD startup scripts
     * Intel Gigabit Driver: wx desupported
     * jp.FreeBSD.org daily SNAPSHOTs project
     * jpman project
     * KSEs
     * LOMAC Status Report
     * Network interface cloning and modularity
     * New mount(2) API
     * NEWCARD/OLDCARD Status report
     * Pluggable Authentication Modules
     * Ports Manager Team (portmgr)
     * RELNOTESng
     * Revised {mode,log}page support for camcontrol
     * SMPng Status Report
     * Status Report: mb_alloc (-CURRENT mbuf allocator)
     * TCP Performance Improvements
     * TrustedBSD Audit
     * TrustedBSD Project
     * UDF Filesystem
     * Web site conversion to XML

ATA Project Status Report

   Contact: So/ren Schmidt <s...@FreeBSD.org>

   Work is underways to support failing mirror disks better and handle
   hotswapping in a new replacement disk and have it rebuild automagically.

   Support for the Promise TX4 is now working in my lab, seems they did the
   PCI-PCI bridging in the not so obvious way.

   Plans are in the works to backport the -current ATA driver to -stable with
   hotswap and the works. Now that -current is delayed I'm working on ways to
   give me time to get this done, since I've had lots of requests lately and
   we really can't let down our customers :).

   SMART support is being worked on, but no timelines yet.

   Although not strictly ATA, Promise has equipped me with a couple SuperTrak
   sx6000 RAID controllers, they take 6 ATA disks and does RAID0-5 in
   hardware. I have done a driver (its an I2O device) for both -current and
   -stable and it works butifully with hotswap the works. It will enter the
   tree when it is more mature, and I have an agreement with Promise on how
   we handle userland control util etc. BTW it seems it can also be used as a
   normal 6 channel PCI ATA controller, a bit on the expensive side maybe...

     ----------------------------------------------------------------------

Device Polling

   URL: http://info.iet.unipi.it/~luigi/polling/

   Contact: Luigi Rizzo <lu...@iet.unipi.it>

   This work uses a mixed interrupt-polling architecture to handle network
   device drivers, giving the system substantial improvements in terms of
   stability and robustness to overloads, as well as the ability to control
   the sharing of CPU between network-related kernel processing and other
   user/kernel tasks. Last not least, you might even see a moderate (up to
   20-30%, machine dependent) performance improvement.

     ----------------------------------------------------------------------

Fibre Channel Support

   URL: http://www.feral.com/isp.html

   Contact: Matthew Jacob <mja...@feral.com>

   Ongoing bug fixes. Work is underway, to be integrated shortly, that makes
   the cross platform endian support easier and will prepare the FreeBSD
   version for eventual sparc64 and PowerPC usage.

     ----------------------------------------------------------------------

FreeBSD 4.5 Release Engineering

   URL: http://www.freebsd.org/internal/releng.html
   URL: http://www.FreeBSD.org/internal/releng45.html

   Contact: Murray Stokely <mur...@FreeBSD.org>

   Release engineering activities for FreeBSD 4.5 have begun. An overview of
   the entire process has been added to the FreeBSD web site, along with a
   specific schedule for 4.5. The code freeze is scheduled to start on
   December 20. The team responsible for responding to MFC requests sent to
   r...@FreeBSD.org for this release is: Murray Stokely, Robert Watson, and
   John Baldwin. Some of our many goals for this release include closing more
   installation-related problem reports, being more conservative with our
   approval of changes during the code freeze, and continuing to document the
   entire process. For suggestions or questions about FreeBSD 4.5 release
   activities, please subscribe to the public freebsd...@FreeBSD.org mailing
   list.

     ----------------------------------------------------------------------

FreeBSD C99 & POSIX Conformance Project

   URL: http://people.FreeBSD.org/~mike/c99/

   Contact: Mike Barcroft <m...@FreeBSD.org>
   Contact: FreeBSD-Standards Mailing List <standa...@FreeBSD.org>

   Work on the FreeBSD C99 & POSIX Conformance Project is progressing nicely.
   Since the last status report, two new headers have been added [<stdint.h>
   and <inttypes.h>], several new functions implemented [atoll(3),
   imaxabs(3), imaxdiv(3), llabs(3), lldiv(3), strerror_r(3), strtoimax(3),
   and strtoumax(3)], and changes to assert(3) and printf(3) were made to
   support C99. More printf(3) changes are in the works to support the
   remaining C99 and POSIX requirements. Additionally, research was done into
   our POSIX Utility conformance and a list of tasks was derived from that
   research.

   Several other interesting events occurred during November and the
   beginning of December. The project mailing list was moved to the
   FreeBSD.org domain, and is now available at standa...@FreeBSD.org. On
   December 6, 2001, the IEEE Standards Board approved the Austin Group
   Specification as IEEE Std 1003.1-2001, thus making the work we're doing
   ever more important.

     ----------------------------------------------------------------------

FreeBSD in Bulgarian

   URL: http://www.FreeBSD-bg.ringlet.net/
   URL: http://people.FreeBSD.org/~roam/bg/

   Contact: Peter Pentchev <r...@FreeBSD.org>

   The FreeBSD in Bulgarian project aims to bring a more comfortable working
   environment to Bulgarian users of the FreeBSD OS. This includes, but is
   not limited to, font, keymap and locale support, translation of the
   FreeBSD documentation into Bulgarian, local user groups and various forms
   of on-line help channels and discussion forums to help Bulgarians adopt
   and use FreeBSD.

   Bulgarian locale support has been committed to FreeBSD 5.0-CURRENT (and
   later merged into 4.x-STABLE on December 10th). A local CVS repository for
   the translation of the FreeBSD documentation into Bulgarian has been
   created.

     ----------------------------------------------------------------------

FreeBSD NVIDIA Driver Port

   URL: ftp://ftp.jurai.net/users/winter/nvidia/NEWS
   URL: ftp://ftp.jurai.net/users/winter/nvidia/

   Contact: Matthew N. Dodd <md...@FreeBSD.org>

   The port of the driver is around 90% feature complete. AGP support and
   "Registry" support via sysctl need to be finished/implemented. The NVIDIA
   guys are working on a build of the X11 libs and extensions for FreeBSD;
   once this is done hardware accelerated direct rendering should work. The
   previous version this driver is no longer available. I'm planning on
   making a snapshot of my code once I chase out a few more bugs.

   Please note that development is taking place under -CURRENT right now; a
   port to -STABLE will be available at some later time.

     ----------------------------------------------------------------------

GEOM - generalized block storage manipulation

   URL: http://www.freebsd.org/~phk/Geom/

   Contact: Poul-Henning Kamp <p...@FreeBSD.org>

   This project is now finally underway, thanks to DARPA and NAI getting a
   sponsorship lined up. The infrastructure code and data structures are
   currently taking form inside a userland simulation harness.

     ----------------------------------------------------------------------

Improving FreeBSD startup scripts

   URL: http://groups.yahoo.com/group/FreeBSD-rc/
   URL: http://www.cs.rmit.edu.au/~lukem/bibliography.html
   URL: http://www.netbsd.org/Documentation/rc/

   Contact: Doug Barton Commiter <Do...@FreeBSD.org>
   Contact: Gordon Tetlow Contributor <gord...@gnf.org>

   <-- from http://groups.yahoo.com/group/FreeBSD-rc/ -->

   This group is for discussion about the startup scripts in FreeBSD,
   primarily the scripts in /etc/rc*. Primary focus will be on improvements
   and importation of NetBSD's excellent work on this topic.

   <-- from Gordon Tetlow's ranting -->

   Due to personal commitments by the folks working on this project we have
   been unable to spend much time porting the rc.d infrastructure into the
   FreeBSD boot framework.

   Currently, the system will boot (with a little fudging) just before
   network utilization. There are patches floating around for this (see the
   -arch list from September).

     ----------------------------------------------------------------------

Intel Gigabit Driver: wx desupported

   Contact: Matthew Jacob <mja...@feral.com>

   The wx driver is desupported and removed from -current. No further support
   for wx in -stable is planned. Newer and better drivers are now in the
   tree.

     ----------------------------------------------------------------------

jp.FreeBSD.org daily SNAPSHOTs project

   URL: http://snapshots.jp.FreeBSD.org/
   URL: ftp://snapshots.jp.FreeBSD.org/pub/FreeBSD/snapshots/i386/

   Contact: Makoto Matsushita <matus...@jp.FreeBSD.org>

   jp.FreeBSD.org daily SNAPSHOTs project is yet another snapshots server
   that provides latest 4-stable and 5-current distribution. You also find
   installable ISO image, live filesystem, HTMLed source code with search
   engine, and more; please check project webpage for more details.

     ----------------------------------------------------------------------

jpman project

   URL: http://www.jp.FreeBSD.org/man-jp/

   Contact: Kazuo Horikawa <horik...@FreeBSD.org>

   Targeting 4.5-RELEASE, we continued to revising
   doc/ja_JP.eucJP/man/man[1256789] to catch up with RELENG_4. Section 3
   updating has 45% finished.

     ----------------------------------------------------------------------

KSEs

   URL: http://www.freebsd.org/~julian/
   URL: http://www.freebsd.org/~jasone/kse/

   Contact: Julian Elischer <jul...@FreeBSD.org>

   I have been working behind the scenes on design rather than programming
   for this last month. I have been working however in the p4 tree to make
   the system run with the thread structure NOT a part of the proc structure
   (a prerequisite fo threading)

     ----------------------------------------------------------------------

LOMAC Status Report

   URL: http://opensource.nailabs.com/lomac/

   Contact: Brian Feldman <gr...@FreeBSD.org>

   A FreeBSD -CURRENT snapshot with LOMAC is currently being prepared, with
   aid of Perforce on the "green_lomac" branch. Very soon there should be a
   working demonstration installation CD of FreeBSD with LOMAC, including the
   ability to enable LOMAC in rc.conf with sysinstall, being a legitimate
   "out-of-the-box" FreeBSD experience. Actual release build is pending
   debugging issues with program start-up (especially xdm).

     ----------------------------------------------------------------------

Network interface cloning and modularity

   Contact: Brooks Davis <bro...@FreeBSD.org>

   Support for VLAN cloning has been merged from current and will ship with
   4.5-RELEASE. Additionaly, new rc.conf support for cloning interfaces at
   boot has been MFD'd. Work is ongoing to MFC stf and faith cloning as well
   as adding cloning for ppp devices and enhancing VLAN modularity.

     ----------------------------------------------------------------------

New mount(2) API

   URL: http://www.sneakerz.org/~mux/mount.diff

   Contact: Poul-Henning Kamp <p...@FreeBSD.org>
   Contact: Maxime Henrion <m...@qualys.com>

   There is now some code ready for the new mount API, which has to be
   reviewed and tested. If it is adopted, we will probably start converting
   all the filesystems, as well as other code in the kernel, to make them use
   it. If you want to play with it, the patch is available at the above URL.

     ----------------------------------------------------------------------

NEWCARD/OLDCARD Status report

   Contact: Warner Losh <i...@FreeBSD.org>

   Not much to report. A number of minor bugs in OLDCARD have been corrected.
   A larger number of machines now work. Additional work on ToPIC support has
   been committed, but continued lack of a suitable ToPIC machine has left
   the author unable to do much work. A few stubborn machines still need to
   be supported (the author has an example of one such machine, so there is
   hope for it being fixed. Some pci related issues remain for both OLDCARD
   and NEWCARD.

   NEWCARD work is ramping up, while OLDCARD work is ramping down. A number
   of things remain to be done for NEWCARD, including suspend/ resume
   support, generic device arrival/removal daemon and hopefully automatic
   loading of drivers. A number of current pccard drivers still need to be
   converted to NEWBUS. Several Chipset issues remain, as does the merging of
   isa pccard bridge code with the pccbb code.

     ----------------------------------------------------------------------

Pluggable Authentication Modules

   URL: http://people.freebsd.org/~des/diary/2001.html

   Contact: Mark Murray <ma...@FreeBSD.org>
   Contact: Dag-Erling Smo/rgrav <d...@FreeBSD.org>

   On the code side, a number of libpam bugs have been fixed; a new PAM
   module, pam_self(8) , has been written; and preparations have been made
   for transitioning from /etc/pam.conf to /etc/pam.d .

   On the documentation side, new manual pages have been written for
   pam_ssh(8) , pam_get_item(3) and pam_set_item(3) , and work has started on
   a longer article about PAM which is expected to be finished by the end of
   the year.

   A lot of work still remains to be done to integrate PAM more tightly with
   the FreeBSD base system-particularly the passwd(1) , chpass(1) etc.
   utilities-and ports collection.

     ----------------------------------------------------------------------

Ports Manager Team (portmgr)

   URL: http://bento.FreeBSD.org/

   Contact: Will Andrews <w...@FreeBSD.org>

   After a discussion with the Core Team about our status regarding the ports
   collection, we heard from them that they'd decided to recognize us as the
   final authority for approving ports committers. We've spent the last few
   weeks working on our ports build cluster (see the link) and trying to find
   ways to improve it for the ports development community. We've also handled
   a few minor issues in the ports collection.

     ----------------------------------------------------------------------

RELNOTESng

   URL: http://people.FreeBSD.org/~bmah/relnotes/
   URL: http://www.FreeBSD.org/relnotes.html

   Contact: Bruce Mah <b...@FreeBSD.org>

   I've been working on making the Hardware Notes less i386-centric. This
   will be especially important for -CURRENT as the ia64 and sparc ports
   reach maturity; most of this work should be completed in time to be MFC-ed
   for FreeBSD 4.5-RELEASE. I encourage any interested parties to review the
   release documentation and send me comments or patches.

     ----------------------------------------------------------------------

Revised {mode,log}page support for camcontrol

   Contact: Kelly Yancey <kby...@FreeBSD.org>

   Extending camcontrol's page definition file format to include both
   modepage and logpage definitions; adding support to camcontrol to query
   and reset log page parameters. Consideration is being made to possibly
   include support for diagnostic and vital product data pages, but that is
   outside the current project scope. New page definition file format
   includes capability to conditionally include page definitions based on
   SCSI INQUIRY results allowing vendor-specific pages to be described also.
   Approximately 80% complete.

     ----------------------------------------------------------------------

SMPng Status Report

   URL: http://www.FreeBSD.org/smp/

   Contact: John Baldwin <j...@FreeBSD.org>
   Contact: <s...@FreeBSD.org>

   October ended up being a bit busier than November for SMPng. During
   October, Peter Wemm finally finished the ambitious task of unwinding all
   the macros in NFS and splitting it up into two halves: client and server.
   Andrew Reiter also submitted some code to add locks to taskqueues, and the
   folks working on the TTY subsystem designed the locking strategy they will
   be using. Per-thread ucred references were also added for user traps and
   syscalls. Once the necessary locking on the process ucred references is
   committed, this will allow kernel code to access the credentials of the
   current thread without needing locks while also ensuring that a thread has
   constant credentials for the lifetime of a syscall. November only saw a
   few small bug fixes unfortunately, but December is already shaping up to
   be a very active month, so next month's report should be a bit more
   interesting.

   In non-coding news, the website for the SMPng project has moved from its
   old location to the new location above. Also, I have completed a paper I
   am presenting for BSDCon regarding the SMPng project. The paper will be
   available in the conference proceedings and will be available online after
   the conference as well.

     ----------------------------------------------------------------------

Status Report: mb_alloc (-CURRENT mbuf allocator)

   URL: http://www.FreeBSD.org/~bmilekic/code/mb_alloc/

   Contact: Bosko Milekic <bmile...@FreeBSD.org>

   Presently re-style(9)ing mbuf code with the help of Bruce (bde). The next
   larger step is approaching: to better performance, as initially planned,
   not have reference counters for clusters allocated separately via
   malloc(9). Rather, use some of the [unused] space at the end of each
   cluster as a counter; since this space is totally unused and since ref.
   counter <--> mbuf cluster is a one-to-one relationship, this is most
   convenient.

     ----------------------------------------------------------------------

TCP Performance Improvements

   Contact: Matthew Dillon <dil...@FreeBSD.org>

   A number of serious TCP bugs effecting throughput snuck into the system
   over the last few releases and have finally been fixed. TCP performance
   should be greatly improved for a number of cases, including TCP/NFS.

     ----------------------------------------------------------------------

TrustedBSD Audit

   URL: http://www.TrustedBSD.org/

   Contact: John Doe <trustedbsd-au...@trustedbsd.org>

   Currently, we are exploring a variety of strategies to learn about the
   implementation and performance issues in order to have a solid design. One
   of our main goals will be to use a standardized interface to the system,
   whether it be POSIX.1e, or another of the other standards, because as they
   say "Standards are great because you have so many to choose from."
   Hopefully within the next month or so, we will populate the perforce
   TrustedBSD tree with an agreed upon framework that is ready for serious
   final work.

     ----------------------------------------------------------------------

TrustedBSD Project

   URL: http://www.TrustedBSD.org/

   Contact: Robert Watson <rwat...@FreeBSD.org>

   The TrustedBSD Project continued focussing development efforts on
   fine-grained Capabilities and Mandatory Access Control this month. Kernel
   support for capabilities is essentially complete, and efforts are underway
   to adapt userland applications to use Capabilities. The login process has
   been updated to allow users to run with additional privilege based on
   /etc/capabilities. The MAC implementation work has also been active, with
   improved support for the labeling of IPC objects, including better
   integration into the network stack. Both development trees have been
   updated to work with recent KSE-related developments, as well as exist
   more happily in a fine-grained SMP kernel. Initial audit-related work
   appears in a seperate entry.

   Development of TrustedBSD source code was moved to the FreeBSD Perforce
   repository, permitting better source code management. As such, the
   TrustedBSD development trees will now be available via cvsup.

     ----------------------------------------------------------------------

UDF Filesystem

   URL: http://people.freebsd.org/~scottl/udf

   Contact: Scott Long <sco...@FreeBSD.org>

   Modest gains have been made on the UDF filesystem since the last report.
   Reading of files from DVD-ROM now works (and is fast, according to some
   reports), and there is preliminary support for reading from CD-RW media.
   The CD-RW support has only been tested against CD's created with Adaptec/
   Roxio DirectCD, and much, much more testing is needed. Once this support
   is solid, I plan to check it into the tree and start work on making the
   filesystem writable.

     ----------------------------------------------------------------------

Web site conversion to XML

   Contact: Nik Clayton <n...@FreeBSD.org>

   Work is (slowly) progressing on converting the web site to use pages
   marked up in a simple XML schema, and then generating HTML and other
   output formats using XSLT style sheets. The work so far can be tested by
   doing "cvs checkout -r XML_XSL_XP www" and then "cd www/en; make
   index.html". Take a look at index.page in the same directory to see the
   source XML. The CVS logs for index.page contain detailed instructions
   explaining how index.page was generated from its earlier form.



To Unsubscribe: send mail to majord...@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message