Path: gmdzi!unido!mcsun!uunet!usenix!std-unix
From: j...@usenix.org
Newsgroups: comp.std.unix
Subject: Standards Update, IEEE 1003.2: Shell and tools
Message-ID: <378@usenix.ORG>
Date: 21 Jun 90 20:17:50 GMT
Sender: std-u...@usenix.ORG
Reply-To: std-u...@uunet.uu.net
Lines: 273
Approved: jsq@usenix (Moderator, John Quarterman)
Posted: Thu Jun 21 21:17:50 1990
From: < j...@usenix.org>
An Update on UNIX*-Related Standards Activities
June 1990
USENIX Standards Watchdog Committee
Jeffrey S. Haemer, Report Editor
IEEE 1003.2: Shell and tools
Randall Howard < r...@mks.com> reports on the April 23-27 meeting in
Salt Lake City, UT:
Background on POSIX.2
The POSIX.2 standard deals with the shell programming language and
utilities. Currently, it is divided into two pieces:
+ POSIX.2, the base standard, deals with the basic shell
programming language and a set of utilities required for
application portability. Application portability essentially
means portability of shell scripts and thus excludes most
features that might be considered interactive. In an analogy to
the ANSI C standard, the POSIX.2 shell command language is the
counterpart to the C programming language, while the utilities
play, roughly, the role of the C library. POSIX.2 also
standardizes command-line and function interfaces related to
certain POSIX.2 utilities (e.g., popen, regular expressions,
etc.) This document is also known as ``Dot 2 Classic.''
+ POSIX.2a, the User Portability Extension or UPE, is a supplement
to the base POSIX.2 standard; it will eventually be an optional
chapter of a future draft of the base document. The UPE
standardizes commands, such as screen editors, that might not
appear in shell scripts but are important enough that users must
learn them on any real system. It is essentially an interactive
standard that attempts to reduce retraining costs incurred by
system-to-system variation.
Some utilities, have interactive as well as non-interactive
features. In such cases, the UPE defines extensions from the
base POSIX.2 utility. An example is the shell, for which the UPE
defines job control, history, and aliases. Features used both
interactively and in scripts tend to be defined in the base
standard.
__________
* UNIX is a registered trademark of AT&T in the U.S. and other
countries.
June 1990 Standards Update IEEE 1003.2: Shell and tools
- 2 -
Together Dot 2 Classic and the UPE will make up the International
Standards Organization's IS 9945/2 - the second volume of the proposed
ISO four-volume standard related to POSIX.
In addition to providing current information about the activities of
the Working and Balloting Groups for POSIX.2, a special topic of focus
will be chosen for each report. Therefore, the reader is referred to
earlier reports for information on such topics as the history of the
Shell Wars and the controversial scope of the UPE. The next section
talks about the functions, rather than utilities, that are found with
POSIX.2.
The POSIX.2 API Functions
Perhaps it will come as a surprise to many readers that the POSIX
Shell and Utilities standard also contains specifications for about
fourteen new or extended C function bindings -- in effect, its own API
extending the POSIX.1 bindings -- as follows:
confstr(), sysconf() The first function was created to provide
string-valued, configuration-specific values such as the
default setting of the PATH environment variable. The
second extends the POSIX.1 function of the same name with
numeric-valued configuration information such as the
largest scale value in the bc utility and the
implementation's line length restriction.
fnmatch() This functional interface implements the form of pattern
matching used by file-name generation (glob) in the shell,
case statements in the shell, and the -name option of the
find utility.
getopt() This functional interface provides a standard utility
argument parser that enforces the ``standard utility
syntax'' guidelines and might be used to implement the
getopts utility from POSIX.2.
glob(), globfree() This set of functions does shell-style file-name
generation and presumably calls the fnmatch() function.
popen(), pclose() This pair of functions, which is a part of the
standard I/O package on conventional UNIX systems,
provides the ability to communicate through pipes to
another process by executing a string in the POSIX.2 shell
language.
regexec(), regcomp() This set of routines provides support for both
the Basic and Extended Regular Expressions defined in
POSIX.2, including full internationalization support.
June 1990 Standards Update IEEE 1003.2: Shell and tools
- 3 -
wordexp(), wordfree() This set of routines provides a mechanism for
an application to use word expansion (parameter expansion)
that is compatible with the POSIX.2 shell language.
Although most implementations of this routine will
normally call the shell, it is (at least conceptually)
possible that the shell be implemented to call these
routines for word expansion.
system() This ``classical'' function executes a command written in
shell language.
All of these functions form part of an optional C binding to POSIX.2
and it is expected that the soon-to-be-released, draft version of the
NIST FIPS will make this ``optional'' functional interface mandatory
for US government procurements. Other language-binding working
groups, such as those exploring Ada and FORTRAN, are presumably
encouraged to add their own optional bindings if they so wish.
Although the inclusion of these functions seems to be a little out of
place in a shell-and-tools standard, there is some rationale for this.
In fact, when POSIX consisted only of POSIX.1, the early attempts to
define system() and popen() made apparent the need to completely
specify the shell language in which the argument string to these
functions was written. That, in turn, along with the desire to
standardize the classical UNIX utility set, led to the creation of
POSIX.2 as the first offshoot group in the POSIX family of standards.
From this beginning, the POSIX.1 sysconf() function was extended and
the confstr() function was created to provide an underlying
implementation for the getconf utility, which allowed shell-level
applications to query configuration-specific values such as maximum
line length of text files. Once the beachhead of having functional
interfaces in POSIX.2 was established, the temptation to continually
add to this list has led to the current list as of Draft 9.
On the other hand, there are some very strong arguments against the
inclusion of these functions. First, although the regular expression
functions will almost certainly be required to implement many POSIX.2
utilities such as ed, grep, awk, sed, etc., these functions stop short
of the complete support needed to implement some utilities. For
example, the handling of error messages (as in a syntactically
incorrect regular expression) and the mechanisms of doing
substitutions (including & and \n support) are not addressed. Because
of this most implementors will be required to have ``non-portable,''
proprietary extensions to their regular expression support to make a
``commercially-viable'' implementation. The issue of where to ``draw
the line'' between inclusion and exclusion is a difficult one indeed.
Second, vendors and application writers may find it difficult, both
procedurally and from a licensing perspective, to have part of the
subroutine library come from a POSIX.1 developer and the other part
implemented by the POSIX.2 implementor. For example, the implementor
of sysconf(), popen(), or system() might do a much better job if
June 1990 Standards Update IEEE 1003.2: Shell and tools
- 4 -
common source code and assumptions were possible between the POSIX.1
and POSIX.2 APIs.
Status of POSIX.2 Balloting
``Dot 2 Classic'' remains in its second round of balloting on Draft 9
with a new draft going to ballot in the June-to-July time frame,
according to Hal Jespersen the POSIX.2 Technical Editor.
During the Snowbird meeting, much of Monday was devoted to a
presentation on the status of the Dot 2 Classic Balloting resolution.
It is possible, and indeed likely, that Hal Jespersen will limit
balloting on Draft 10 to unresolved objections and new material. If
this is the case, it most likely indicates (although he didn't
specifically say) that Hal has confidence that Draft 10 has a high
probability of achieving the requisite 75% affirmative vote.
Personally, I am not convinced that this is a likely event. While
some decisions will be reversed (perhaps several times) before Draft
10, the following is a summary of issues and/or changes appearing in
Draft 10:
+ The internationalization utilities locale and particularly
localedef are still controversial, particularly within AT&T.
Because of the strong rationale for their existence it appears
that they will remain in Draft 10, certainly with considerable
amendment as the UniForum Technical Committee on
Internationalization refines these newly developed utilities.
This is just one case where the conflict between the role of
standards to codify existing practice and the obvious holes in
existing practice creates controversy. Perhaps this issue will
be resolved by a balloting referendum such as was used for uucp.
+ The Draft 10 shell will almost certainly strongly resemble that
of Draft 9. Most of the important controversies appear to be
largely settled and most changes appear to be corrections and
clarifications.
+ Most complex utilities, such as awk, shell, lex, yacc, etc., have
undergone extensive reworking in response to ballot objections.
Often a seemingly simple objection will cause large parts of the
description to be rewritten in order to tighten it up with
respect to completeness and clarity. I believe that Hal
Jespersen believes that most of these changes are uncontroversial
and he has ensured this by circulating draft sections via E-mail
to various ``experts.'' Certainly, many of these utilities
desperately needed this clarification.
+ It appears that the newly-engineered hexdump utility is to be
replaced by a (much simpler) reversion to od. While od is the
existing practice, the POSIX od will be a superset of the
original one with most useful functionality in the new parts. It
June 1990 Standards Update IEEE 1003.2: Shell and tools
- 5 -
is not clear that hiding new invention under the same name is any
less controversial than advertising its existence.
+ Of course, there will be innumerable other changes, obviously
important to many, that cannot for reasons of space be covered
here.
A mock ballot on Draft 4 of the UPE was sent to the working group
during February 1990 to allow ballot resolution to be the main focus
of the Salt Lake meeting this April.
Status of the New Orleans Meeting
Monday, the working group reviewed the current status of the balloting
on ``Dot 2 Classic''. This has already been discussed in earlier in
this report.
The other four days were spent reviewing the 600 to 700 objections
produced by the mock balloting process for the UPE. While the number
of objections seems low compared to the rate of objections for the
corresponding number of pages in Dot 2 Classic, this may simply be a
symptom of a general shortage of time and the lower number of people
(generally 15 to 20) in the UPE working group. This lower number and
general lack of time, is a reflection of the fragmentation of the
entire POSIX process caused by a proliferation of working groups.
Most of the work during mock balloting was of the nature of cleaning
up incomplete or poorly worded textual descriptions. Particularly
controversial issues were often left in the rationale for Draft 5.
Some controversial utilities were moved to an appendix based upon the
belief that they should be removed while still allowing the balloting
group one last chance to save them. The lint89 was one such utility
whose raison d'etre was meager. At best, the functionality probably
should be an option to c89 in the ``Dot 2 Classic'' document. The
sdiff utility which was inadvertently omitted from Draft 4, is to be
included in Draft 5.
Altogether, it appears that Draft 5 is in a relatively healthy state
to survive the rigors of the balloting process. None the less, I
expect that there will be a greater number of objections in the
balloting this summer than there were in the mock ballot.
June 1990 Standards Update IEEE 1003.2: Shell and tools
Volume-Number: Volume 20, Number 35
|