Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!henry
From: he...@utzoo.UUCP (Henry Spencer)
Newsgroups: news.software.b
Subject: C news bulletin #1
Message-ID: <8918@utzoo.UUCP>
Date: Thu, 12-Nov-87 12:56:24 EST
Article-I.D.: utzoo.8918
Posted: Thu Nov 12 12:56:24 1987
Date-Received: Thu, 12-Nov-87 12:56:24 EST
Organization: U of Toronto Zoology
Lines: 33
Several people have noticed that some of the Makefiles invoke a mysterious
program "makedtr", which is nowhere in evidence. Sorry about that. The
problem is that since the alpha release was a clone of our working copies,
rather than a polished distribution, you got the stuff that was meant for
assembling the distribution rather than the result. Makedtr is just a
simple shell archiver with a special tweak or two. It plays no particularly
vital part in things, except that the batcher Makefile wants to use it to
prepare a file of test data. You can just twiddle the Makefile to ignore
the test stuff, but if you don't want to do that, here is makedtr:
----------
for f
do
case "$f"
in
*Makefile)
name=Makefile
;;
*)
name="$f"
;;
esac
echo "echo '$name':"
echo "sed 's/^X//' >'$name' <<'!'"
sed 's/^/X/' "$f"
echo "!"
done
echo "echo done"
----------
--
Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology
condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry
Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!henry
From: he...@utzoo.UUCP (Henry Spencer)
Newsgroups: news.software.b
Subject: C News Bulletin #2 - three fixes to expire subsystem
Message-ID: <8956@utzoo.UUCP>
Date: Tue, 17-Nov-87 20:58:53 EST
Article-I.D.: utzoo.8956
Posted: Tue Nov 17 20:58:53 1987
Date-Received: Tue, 17-Nov-87 20:58:53 EST
Organization: U of Toronto Zoology
Lines: 14
The following are fixes for things in the expire directory:
1. In upact, "umask NEWSUMASK" should be "umask $NEWSUMASK".
2. In expire.c, in the function control(), if you put "if (ctl[0] != '#')"
in front of "ctlline(ctl);", you will be able to put #-comment lines
into the expiry control file, which is a Good Thing if you have
complex expiry policies.
3. In expire.c, in the function printstuff(), "field[1]" should be "field[0]".
You will notice a difference only if you use the -p option.
--
Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology
condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry
Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!henry
From: he...@utzoo.UUCP (Henry Spencer)
Newsgroups: news.software.b
Subject: C News Bulletin #3 - important fix to history rebuild
Message-ID: <8971@utzoo.UUCP>
Date: Sat, 21-Nov-87 22:19:09 EST
Article-I.D.: utzoo.8971
Posted: Sat Nov 21 22:19:09 1987
Date-Received: Sat, 21-Nov-87 22:19:09 EST
Organization: U of Toronto Zoology
Lines: 22
For a while during its development, C news used a 4-field history file.
We changed that to the current form, using a multi-part third field, to
avoid fouling up news readers that understood the file format. We (well,
specifically, I) missed one place that needed to be fixed to match this
format change. In the expire subsystem, the histdups program is used to
merge cross-postings during rebuilding of the history file. It needs the
following fix:
12c12
< names = names " " $4
---
> names = names " " $3
Geoff spotted this one. The effect of using histdups as distributed is
that when the history file is rebuilt, cross-posted articles get into it
under their first name only. The result is a buildup of old articles, ones
that have been expired under their first name but are lingering on under
other names. To clear up the problem, make the fix, rebuild your history
file one more time, and do an expire run.
--
Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology
condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry
Relay-Version: version B 2.10 5/3/83; site utzoo.UUCP
Posting-Version: version B 2.10 5/3/83; site utzoo.UUCP
Path: utzoo!henry
From: he...@utzoo.UUCP (Henry Spencer)
Newsgroups: news.software.b
Subject: C News Bulletin #4 - minor expire fix, and some Q&A
Message-ID: <9109@utzoo.UUCP>
Date: Mon, 14-Dec-87 19:06:48 EST
Article-I.D.: utzoo.9109
Posted: Mon Dec 14 19:06:48 1987
Date-Received: Mon, 14-Dec-87 19:06:48 EST
Organization: U of Toronto Zoology
Lines: 51
Thanks to Jim Rees at Apollo for finding this one:
In expire.c, checkexpire(), about line 561, this line
if (*scan == NULL) {
should read
if (scan == NULL) {
This didn't show up in testing because it matters only if scan is NULL,
which happens only with a messed-up history file.
Also, herewith some answers to some of the more interesting questions that
have been asked:
Q: Expire is awfully touchy about file formats etc, and makes no attempt
to recover. Is this right?
A: Expire's business is to destroy files. This is best done cautiously.
If something is wrong, appealing for human intervention rather
than pressing on is at least a reasonable approach.
Q: The batcher would be lots faster if written in C, wouldn't it?
A: Not much. Its run time is dominated by the "batchmake|compress|uux"
pipeline, which is already entirely in C. Keeping the rest of
the code in shell and awk doesn't hurt speed much, and makes
maintenance and customizing much easier.
Q: There are a lot of shell files between newsrun and relaynews, aren't
there? This whole area seems confused and overly complex. Could
it not be simplified and speeded up?
A: Yes, in fact we've done this, although it's not ready for release yet.
The multiple layers of shell file there were partly historical
artifact and partly an artifact of the way the development was
divided up (I did the input subsystem, Geoff did relaynews etc.).
You should have seen what a mess it was when almost every program
involved was named "rnews"... We spent a noticeable amount of
time cleaning up the naming, just before distribution; unfortunately
we didn't have the time to clean up the software too.
Q: It is very inconvenient that the incoming-spool area is under /usr/lib/news
rather than /usr/spool/news. Can this be changed?
A: It's not entirely straightforward; it would break history rebuilding, in
particular. However, we agree it is desirable and it will be done.
--
Those who do not understand Unix are | Henry Spencer @ U of Toronto Zoology
condemned to reinvent it, poorly. | {allegra,ihnp4,decvax,utai}!utzoo!henry
Newsgroups: news.software.b
Path: utzoo!utstat!geoff
From: ge...@utstat.uucp (Geoff Collyer)
Subject: C News Bulletin #5 - rnews bugs and caveats
Message-ID: <1987Dec16.025423.18295@utstat.uucp>
Organization: Statistics, U. of Toronto
Date: Wed, 16-Dec-87 02:54:22 EST
The C news alpha release contains bugs and bad behaviours, as promised
in the README files. Here are some of the more prominent ones in rnews
and friends (most bugs uncovered thus far have been fairly minor).
Chris Lewis found that libc/putenv.c doesn't always return a value.
(Oops, I guess putenv hadn't been linted recently.) The fix is to
change the two occurrences of "return;" to "return YES;". It has been
observed that this putenv's return value differs from the System V
putenv's; the difference will be reconciled before final release.
rnews/fileart.c fails to set errno when appropriate on systems lacking a
3-argument open (e.g. V7). (I never should have let utcsstat, our V7
11/70 go!) Henry found and fixed this in a more recent (and quite
different) form of this code. An equivalent fix to the alpha release
code is to change this code in openlink() starting at about line 238 from
if (access(sp, F_OK) >= 0) /* sp exists */
*tfp = NULL; /* refuse to write on it */
else
to
if (access(sp, F_OK) >= 0) { /* sp exists */
extern int errno;
errno = EEXIST;
*tfp = NULL; /* refuse to write on it */
} else
Alan Hargreaves found a few typos: libc/makefile refers to
libc/memlist.c; just delete the reference. rnews/system.h defines
FLG_IHAVE and FLG_IHAVEOLD, which are not unique in their first nine
characters; Alan's fix is to change FLG_IHAVEOLD to FLG_OLDIHAVE
everywhere.
Alan also found that inews does not understand newsgroups of the form
all.all.ctl. There are several possible fixes, but I haven't applied
any of them yet (I feed my control messages to rnews or relaynews
directly, by hand). I plan to emit a "Control:" header cloned from the
"Subject:" line and delete ".ctl", as for "Subject: cmsg foo", since
this form of control message explicitly does not conform to RFC 850
(it's a backwardly compatible hack).
relaynews -p prints a spurious "relaynews: article <msgid> short by
2147482799 bytes" complaint when fed an unbatched article. This is
annoying but harmless. It's caused by overloading -p and will be fixed
by introducing more options to relaynews (yes, a cop-out).
Those are the bugs; now a bit of vague philosophy to ward off flames
(too late, alas).
Aside from a dig at System V for not including dbm(3) (ATTIS folks,
that's a hint), there is no built-in bias in C news towards System V
or 4.2BSD. We have been accused of bias in favour of both(!), which may
be because we use library functions from both (and provide emulations
for those who lack them), thus offending those in each camp.
We intend C news to be portable and it already runs on at least V7,
System V, 4.2BSD, PDP-11s, Suns, VAXen, and Dual 68k boxes. I expect it
will run on V8 and V9, and it will in time. I have tried to keep
#ifdefs out of the code, so porting to another Unix variant (e.g. a
faster SVR3 port) is simply a matter of making up a new vers directory,
populating it, and fixing rnews/makefile (well, you might have to fix up
expire too).
Having said that, C news does use some little-used or unusual features
of Unix, notably close-on-exec file descriptors, exclusive creat
(3-argument open), fork(), and traditional (non-symbolic or ``hard'')
links. A port to Eunice, MS/DOS or CMS is not guaranteed to be easy.
Henry has commented on the mess of shell scripts front-ending relaynews.
The situation is better now, though the shell scripts are still
evolving. It is important to realise that the shell scripts, notably
inews, rnews, serverrnews and realrnews, are meant as prototypes: if
they work for you, great; if not, adapt them to your local needs.
That's the major reason that they *are* shell scripts: so that people
can adapt them without wading into unfamiliar C code. (As one who had
to wade into B news to adapt it to local circumstances, I certainly
appreciate the malleability of shell scripts.)
--
Geoff Collyer utzoo!utstat!geoff, utstat.toronto.{edu,cdn}!geoff
Newsgroups: news.software.b
Path: utzoo!utstat!geoff
From: ge...@utstat.uucp (Geoff Collyer)
Subject: C News Bulletin #6 - printf vs rnews caveat, locking discussion
Message-ID: <1987Dec19.232650.26096@utstat.uucp>
Organization: Statistics, U. of Toronto
Date: Sat, 19 Dec 87 23:26:49 EST
Henry and I simultaneously found a rather nasty bug, or at least
unportability, in the printf family which can result in corrupted active
files. So far we have only seen this problem on PDP-11 V7, but Ian
Darwin found a related problem on his Dual System V 68k box two years
ago (we provide a workaround for that case).
The affected code is in rnews/active.c, in the function incartnum.
It attempts to format an article number as a five-digit string, padded
with zeros on the left, using the format "ldzeropad", defined in
rnews/zeropad.c. The default ldzeropad in the alpha release is
"%0*.*ld". (The ANSI C drafts support this form, though they deprecate
it.)
The distributed rnews/zeropad.c is a link to
rnews/bugs/zeropad/okay/zeropad.c. An alternate zeropad.c for people
with certain kinds of broken printf (or new SVID-compliant printf) is
rnews/bugs/zeropad/bugged/zeropad.c, which fixes the problem that Ian
saw: "%*.*ld".
On PDP-11 V7 (and possibly other implementations), "%0*.*ld" looks like
the correct format, but printf has a bug: if the decimal representation
of the number being formatted exactly fills the field width (five in
this case), printf emits an extra leading zero. This causes the second
field of the active file entry to be an order of magnitude too small(!),
since only the leftmost five characters of that field are stored on
disk. "%*.*ld" doesn't produce the desired output, but "%0*ld" comes
close: it zero-fills to the field width correctly, but can produce more
digits than the field width if the number being formatted exceeds 99999,
again leading to a too-small active file entry.
I concluded that it is simpler (and more robust!) to provide my own
long-to-zero-filled-ascii output conversions than to try to deal with
the growing variety of printf formats needed to do the job, and is
simpler than to distribute my own unfinished, portable printf
implementation. If your printf can't be made to correctly perform the
output conversion, you'll need to replace the sprintf call in
incartnum() with equivalent code; mine is scattered across several
source files and is little big to include in this message, but doing it
yourself is pretty easy.
Has anybody thought about widening the active file fields to more than
five digits? utzoo already has a few newsgroups with five-digit numbers
(around 11000) in their active file entries, and it's only been about a
year since The Great Renaming, when all the entries were created afresh.
Several people have asked why C news does such simple locking, using
links and not removing "obviously stale" locks. Superior locking
facilities found in most modern Unixes have wildly different interfaces
and function, so no efficient and portable abstract interface presented
itself. Furthermore, some of these facilities do not work over network
file systems (e.g. 4.2BSD's flock system call over Sun's NFS). The uucp
heuristic of checking to see if the process whose process id is stored
in the active file is still alive via kill(pid, 0) (or ps) also falls
down in the presence of network file systems. (I already share
/usr/spool/uucp across all our machines, but I must currently execute
uucp and uux on the file server which owns /usr/spool/uucp, and I wish
I could execute them locally, but that would require rethinking uucp's
stale-lock removal.)
I also like having a lock be an object in the file system name space, so
that it can be manually examined (and possibly removed), rather than a
lock being an invisible attribute of an in-core i-node or a peculiar
invisible entity accessible only through special system calls (shades of
System V IPC!).
Perhaps more importantly, the breakdown of a locking protocol may be
symptomatic of far greater problems, and just forcing the lock and
blindly barging ahead may be the wrong thing to do. From a practical
standpoint, we have found C news to be quite robust and its components,
notably expire and rnews, do not dump core in production, once one has
configured and installed them correctly. (Rnews dumped core on one of
my machines once in production, during development, over a year ago.) A
dedicated opponent possibly could make expire or rnews dump core, but I
am bullet-proofing rnews so that ultimately that should not be possible
either.
--
Geoff Collyer utzoo!utstat!geoff, utstat.toronto.{edu,cdn}!geoff
|