RISCI_ATOM
/
harvey
spiegel van https://github.com/Harvey-OS/harvey.git


			
				
					
						
						
							123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642
							<html>
<title>
data
</title>
<body BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000FF" VLINK="#330088" ALINK="#FF0044">
<H1>Changes to the Programming Environment
<br>
in the
<br>
Fourth Release of Plan 9
</H1>
<DL><DD><I>Rob Pike<br>
<br>&#32;<br>
rob@plan9.bell-labs.com<br>
</I></DL>
<H4>Introduction
</H4>
<P>
The fourth release of Plan 9 includes changes at many levels of the system,
with repercussions in the libraries and program interfaces.
This document summarizes the changes and describes how
existing programs must be modified to run in the new release.
It is not exhaustive, of course; for further detail about any of the
topics refer to the manual pages, as always.
</P>
<P>
Programmers new to Plan 9 may find valuable tidbits here, but the
real audience for this paper is those with a need to update applications
and servers written in C for earlier releases of the Plan 9 operating system.
</P>
<H4>9P, NAMELEN, and strings
</H4>
<P>
The underlying file service protocol for Plan 9, 9P, retains its basic form
but has had a number of adjustments to deal with longer file names and error strings,
new authentication mechanisms, and to make it more efficient at
evaluating file names.
The change to file names affects a number of system interfaces;
because file name elements are no longer of fixed size, they can
no longer be stored as arrays.
</P>
<P>
9P used to be a fixed-format protocol with
<TT>NAMELEN</TT>-sized
byte arrays representing file name elements.
Now, it is a variable-format protocol, as described in
<A href="/magic/man2html/5/intro"><I>intro</I>(5),
</A>in which strings are represented by a count followed by that many bytes.
Thus, the string
<TT>ken</TT>
would previously have occupied 28
(<TT>NAMELEN</TT>)
bytes in the message; now it occupies 5: a two-byte count followed by the three bytes of
<TT>ken</TT>
and no terminal zero.
(And of course, a name could now be much longer.)
A similar format change has been made to
<TT>stat</TT>
buffers: they are no longer
<TT>DIRLEN</TT>
bytes long but instead have variable size prefixed by a two-byte count.
And in fact the entire 9P message syntax has changed: every message
now begins with a message length field that makes it trivial to break the
string into messages without parsing them, so
<TT>aux/fcall</TT>
is gone.
A new library entry point,
<TT>read9pmsg</TT>,
makes it easy for user-level servers to break the client data stream into 9P messages.
All servers should switch from using
<TT>read</TT>
(or the now gone
<TT>getS)</TT>
to using
<TT>read9pmsg</TT>.
</P>
<P>
This change to 9P affects the way strings are handled by the kernel and throughout
the system.
The consequences are primarily that fixed-size arrays have been replaced
by pointers and counts in a variety of system interfaces.
Most programs will need at least some adjustment to the new style.
In summary:
<TT>NAMELEN</TT>
is gone, except as a vestige in the authentication libraries, where it has been
rechristened
<TT>ANAMELEN</TT>.
<TT>DIRLEN</TT>
and
<TT>ERRLEN</TT>
are also gone.
All programs that mention
these constants
will need to be fixed.
</P>
<P>
The simplest place to see this change is in the
<TT>errstr</TT>
system call, which no longer assumes a buffer of length
<TT>ERRLEN</TT>
but now requires a byte-count argument:
<DL><DT><DD><TT><PRE>
char buf[...];

errstr(buf, sizeof buf);
</PRE></TT></DL>
The buffer can be any size you like.
For convenience, the kernel stores error strings internally as 256-byte arrays,
so if you like &#173; but it's not required &#173; you can use the defined constant
<TT>ERRMAX=</TT>256
as a good buffer size.
Unlike the old
<TT>ERRLEN</TT>
(which had value 64),
<TT>ERRMAX</TT>
is advisory, not mandatory, and is not part of the 9P specification.
</P>
<P>
With names, stat buffers, and directories, there isn't even an echo of a fixed-size array any more.
</P>
<H4>Directories and wait messages
</H4>
<P>
With strings now variable-length, a number of system calls needed to change:
<TT>errstr</TT>,
<TT>stat</TT>,
<TT>fstat</TT>,
<TT>wstat</TT>,
<TT>fwstat</TT>,
and
<TT>wait</TT>
are all affected, as is
<TT>read</TT>
when applied to directories.
</P>
<P>
As far as directories are concerned, most programs don't use the system calls
directly anyway, since they operate on the machine-independent form, but
instead call the machine-dependent
<TT>Dir</TT>
routines
<TT>dirstat</TT>,
<TT>dirread</TT>,
etc.
These used to fill user-provided fixed-size buffers; now they return objects allocated
by
<TT>malloc</TT>
(which must therefore be freed after use).
To `stat' a file:
<DL><DT><DD><TT><PRE>
Dir *d;

d = dirstat(filename);
if(d == nil){
	fprint(2, "can't stat %s: %r\n", filename);
	exits("stat");
}
use(d);
free(d);
</PRE></TT></DL>
A common new bug is to forget to free a
<TT>Dir</TT>
returned by
<TT>dirstat</TT>.
</P>
<P>
<TT>Dirfstat</TT>
and
<TT>Dirfwstat</TT>
work pretty much as before, but changes to 9P make
it possible to exercise finer-grained control on what fields
of the
<TT>Dir</TT>
are to be changed; see
<A href="/magic/man2html/2/stat"><I>stat</I>(2)
</A>and
<A href="/magic/man2html/5/stat"><I>stat</I>(5)
</A>for details.
</P>
<P>
Reading a directory works in a similar way to
<TT>dirstat</TT>,
with
<TT>dirread</TT>
allocating and filling in an array of
<TT>Dir</TT>
structures.
The return value is the number of elements of the array.
The arguments to
<TT>dirread</TT>
now include a pointer to a
<TT>Dir*</TT>
to be filled in with the address of the allocated array:
<DL><DT><DD><TT><PRE>
Dir *d;
int i, n;

while((n = dirread(fd, &amp;d)) &gt; 0){
	for(i=0; i&lt;n; i++)
		use(&amp;d[i]);
	free(d);
}
</PRE></TT></DL>
A new library function,
<TT>dirreadall</TT>,
has the same form as
<TT>dirread</TT>
but returns the entire directory in one call:
<DL><DT><DD><TT><PRE>
n = dirreadall(fd, &amp;d)
for(i=0; i&lt;n; i++)
	use(&amp;d[i]);
free(d);
</PRE></TT></DL>
If your program insists on using the underlying
<TT>stat</TT>
system call or its relatives, or wants to operate directly on the
machine-independent format returned by
<TT>stat</TT>
or
<TT>read</TT>,
it will need to be modified.
Such programs are rare enough that we'll not discuss them here beyond referring to
the man page
<A href="/magic/man2html/2/stat"><I>stat</I>(2)
</A>for details.
Be aware, though, that it used to be possible to regard the buffer returned by
<TT>stat</TT>
as a byte array that began with the zero-terminated
name of the file; this is no longer true.
With very rare exceptions, programs that call
<TT>stat</TT>
would be better recast to use the
<TT>dir</TT>
routines or, if their goal is just to test the existence of a file,
<TT>access</TT>.
</P>
<P>
Similar changes have affected the
<TT>wait</TT>
system call.  In fact,
<TT>wait</TT>
is no longer a system call but a library routine that calls the new
<TT>await</TT>
system call and returns a newly allocated machine-dependent
<TT>Waitmsg</TT>
structure:
<DL><DT><DD><TT><PRE>
Waitmsg *w;

w = wait();
if(w == nil)
	error("wait: %r");
print("pid is %d; exit string %s\n", w-&gt;pid, w-&gt;msg);
free(w);
</PRE></TT></DL>
The exit string
<TT>w-&gt;msg</TT>
may be empty but it will never be a nil pointer.
Again, don't forget to free the structure returned by
<TT>wait</TT>.
If all you need is the pid, you can call
<TT>waitpid</TT>,
which reports just the pid and doesn't return an allocated structure:
<DL><DT><DD><TT><PRE>
int pid;

pid = waitpid();
if(pid &lt; 0)
	error("wait: %r");
print("pid is %d\n", pid);
</PRE></TT></DL>
</P>
<H4>Quoted strings and tokenize
</H4>
<P>
<TT>Wait</TT>
gives us a good opportunity to describe how the system copes with all this
free-format data.
Consider the text returned by the
<TT>await</TT>
system call, which includes a set of integers (pids and times) and a string (the exit status).
This information is formatted free-form; here is the statement in the kernel that
generates the message:
<DL><DT><DD><TT><PRE>
n = snprint(a, n, "%d %lud %lud %lud %q",
	wq-&gt;w.pid,
	wq-&gt;w.time[TUser], wq-&gt;w.time[TSys], wq-&gt;w.time[TReal],
	wq-&gt;w.msg);
</PRE></TT></DL>
Note the use of
<TT>%q</TT>
to produce a quoted-string representation of the exit status.
The
<TT>%q</TT>
format is like %s but will wrap
<TT>rc</TT>-style
single quotes around the string if it contains white space or is otherwise ambiguous.
The library routine
<TT>tokenize</TT>
can be used to parse data formatted this way: it splits white-space-separated
fields but understands the
<TT>%q</TT>
quoting conventions.
Here is how the
<TT>wait</TT>
library routine builds its
<TT>Waitmsg</TT>
from the data returned by
<TT>await</TT>:
<DL><DT><DD><TT><PRE>
Waitmsg*
wait(void)
{
	int n, l;
	char buf[512], *fld[5];
	Waitmsg *w;

	n = await(buf, sizeof buf-1);
	if(n &lt; 0)
		return nil;
	buf[n] = ' ';
	if(tokenize(buf, fld, nelem(fld)) != nelem(fld)){
		werrstr("couldn't parse wait message");
		return nil;
	}
	l = strlen(fld[4])+1;
	w = malloc(sizeof(Waitmsg)+l);
	if(w == nil)
		return nil;
	w-&gt;pid = atoi(fld[0]);
	w-&gt;time[0] = atoi(fld[1]);
	w-&gt;time[1] = atoi(fld[2]);
	w-&gt;time[2] = atoi(fld[3]);
	w-&gt;msg = (char*)&amp;w[1];
	memmove(w-&gt;msg, fld[4], l);
	return w;
}
</PRE></TT></DL>
</P>
<P>
This style of quoted-string and
<TT>tokenize</TT>
is used all through the system now.
In particular, devices now
<TT>tokenize</TT>
the messages written to their
<TT>ctl</TT>
files, which means that you can send messages that contain white space, by quoting them,
and that you no longer need to worry about whether or not the device accepts a newline.
In other words, you can say
<DL><DT><DD><TT><PRE>
echo message &gt; /dev/xx/ctl
</PRE></TT></DL>
instead of
<TT>echo</TT>
<TT>-n</TT>
because
<TT>tokenize</TT>
treats the newline character as white space and discards it.
</P>
<P>
While we're on the subject of quotes and strings, note that the implementation of
<TT>await</TT>
used
<TT>snprint</TT>
rather than
<TT>sprint</TT>.
We now deprecate
<TT>sprint</TT>
because it has no protection against buffer overflow.
We prefer
<TT>snprint</TT>
or
<TT>seprint</TT>,
to constrain the output.
The
<TT>%q</TT>
format is cleverer than most in this regard:
if the string is too long to be represented in full,
<TT>%q</TT>
is smart enough to produce a truncated but correctly quoted
string within the available space.
</P>
<H4>Mount
</H4>
<P>
Although strings in 9P are now variable-length and not zero-terminated,
this has little direct effect in most of the system interfaces.
File and user names are still zero-terminated strings as always;
the kernel does the work of translating them as necessary for
transport.
And of course, they are now free to be as long as you might want;
the only hard limit is that their length must be represented in 16 bits.
</P>
<P>
One example where this matters is that the file system specification in the
<TT>mount</TT>
system call can now be much longer.
Programs like
<TT>rio</TT>
that used the specification string in creative ways were limited by the
<TT>NAMELEN</TT>
restriction; now they can use the string more freely.
<TT>Rio</TT>
now accepts a simple but less cryptic specification language for the window
to be created by the
<TT>mount</TT>
call, e.g.:
<DL><DT><DD><TT><PRE>
% mount <I>wsys /mnt/wsys 'new -dx 250 -dy 250 -pid 1234'
</PRE></TT></DL>
In the old system, this sort of control was impossible through the
</I><TT>mount</TT><I>
interface.
</P>
</I><P>
While we're on the subject of
<TT>mount</TT>,
note that with the new security architecture
(see
<A href="/magic/man2html/4/factotum"><I>factotum</I>(4)),
</A>9P has moved its authentication outside the protocol proper.
(For a full description of this change to 9P, see
<A href="/magic/man2html/2/fauth"><I>fauth</I>(2),
</A><A href="/magic/man2html/5/attach"><I>attach</I>(5),
</A>and the paper
<I>Security in Plan 9</I>.)<I>
The most explicit effect of this change is that
</I><TT>mount</TT><I>
now takes another argument,
</I><TT>afd</TT><I>,
a file descriptor for the
authentication file through which the authentication will be made.
For most user-level file servers, which do not require authentication, it is
sufficient to provide
</I><TT>-1</TT><I>
as the value of
</I><TT>afd:</TT><I>
<DL><DT><DD><TT><PRE>
if(mount(fd, -1, "/mnt/wsys", MREPL,
   "new -dx 250 -dy 250 -pid 1234") &lt; 0)
	error("mount failed: %r");
</PRE></TT></DL>
To connect to servers that require authentication, use the new
</I><TT>fauth</TT><I>
system call or the reimplemented
</I><TT>amount</TT><I>
(authenticated mount) library call.
In fact, since
</I><TT>amount</TT><I>
handles both authenticating and non-authenticating servers, it is often
easiest just to replace calls to
</I><TT>mount</TT><I>
by calls to
</I><TT>amount</TT><I>;
see
<A href="/magic/man2html/2/auth"></I><I>auth</I><I>(2)
</A>for details.
</P>
</I><H4>Print
</H4>
<P>
The C library has been heavily reworked in places.
Besides the changes mentioned above, it
now has a much more complete set of routines for handling
<TT>Rune</TT>
strings (that is, zero-terminated arrays of 16-bit character values).
The most sweeping changes, however, are in the way formatted I/O is performed.
</P>
<P>
The
<TT>print</TT>
routine and all its relatives have been reimplemented to offer a number
of improvements:
</P>
<DL COMPACT>
<DT>(1)<DD>
Better buffer management, including the provision of an internal flush
routine, makes it unnecessary to provide large buffers.
For example,
<TT>print</TT>
uses a much smaller buffer now (reducing stack load) while simultaneously
removing the need to truncate the output string if it doesn't fit in the buffer.
<DT>(2)<DD>
Global variables have been eliminated so no locking is necessary.
<DT>(3)<DD>
The combination of (1) and (2) means that the standard implementation of
<TT>print</TT>
now works fine in threaded programs, and
<TT>threadprint</TT>
is gone.
<DT>(4)<DD>
The new routine
<TT>smprint</TT>
prints into, and returns, storage allocated on demand by
<TT>malloc</TT>.
<DT>(5)<DD>
It is now possible to print into a
<TT>Rune</TT>
string; for instance,
<TT>runesmprint</TT>
is the
<TT>Rune</TT>
analog of
<TT>smprint</TT>.
<DT>(6)<DD>
There is improved support for custom
print verbs and custom output routines such as error handlers.
The routine
<TT>doprint</TT>
is gone, but
<TT>vseprint</TT>
can always be used instead.
However, the new routines
<TT>fmtfdinit</TT>,
<TT>fmtstrinit</TT>,
<TT>fmtprint</TT>,
and friends
are often a better replacement.
The details are too long for exposition here;
<A href="/magic/man2html/2/fmtinstall"><I>fmtinstall</I>(2)
</A>explains the new interface and provides examples.
<DT>(7)<DD>
Two new format flags, space and comma, close somewhat the gap between
Plan 9 and ANSI C.
</dl>
<P>
Despite these changes, most programs will be unaffected;
<TT>print</TT>
is still
<TT>print</TT>.
Don't forget, though, that
you should eliminate calls to
<TT>sprint</TT>
and use the
<TT>%q</TT>
format when appropriate.
</P>
<H4>Binary compatibility
</H4>
<P>
The discussion so far has been about changes at the source level.
Existing binaries will probably run without change in the new
environment, since the kernel provides backward-compatible
system calls for
<TT>errstr</TT>,
<TT>stat</TT>,
<TT>wait</TT>,
etc.
The only exceptions are programs that do either a
<TT>mount</TT>
system call, because of the security changes and because
the file descriptor in
<TT>mount</TT>
must point to a new 9P connection; or a
<TT>read</TT>
system call on a directory, since the returned data will
be in the new format.
A moment's reflection will discover that this means old
user-level file servers will need to be fixed to run on the new system.
</P>
<H4>File servers
</H4>
<P>
A full description of what user-level servers must do to provide service with
the new 9P is beyond the scope of this paper.
Your best source of information is section 5 of the manual,
combined with study of a few examples.
<TT>/sys/src/cmd/ramfs.c</TT>
is a simple example; it has a counterpart
<TT>/sys/src/lib9p/ramfs.c</TT>
that implements the same service using the new
<A href="/magic/man2html/2/9p"><I>9p</I>(2)
</A>library.
</P>
<P>
That said, it's worth summarizing what to watch for when converting a file server.
The
<TT>session</TT>
message is gone, and there is a now a
<TT>version</TT>
message that is exchanged at the start of a connection to establish
the version of the protocol to use (there's only one at the moment, identified by
the string
<TT>9P2000</TT>)
and what the maximum message size will be.
This negotiation makes it easier to handle 9P encapsulation, such as with
<TT>exportfs</TT>,
and also permits larger message sizes when appropriate.
</P>
<P>
If your server wants to authenticate, it will need to implement an authentication file
and implement the
<TT>auth</TT>
message; otherwise it should return a helpful error string to the
<TT>Tauth</TT>
request to signal that authentication is not required.
</P>
<P>
The handling of
<TT>stat</TT>
and directory reads will require some changes but they should not be fundamental.
Be aware that seeking on directories is forbidden, so it is fine if you disregard the
file offset when implementing directory reads; this makes it a little easier to handle
the variable-length entries.
You should still never return a partial directory entry; if the I/O count is too small
to return even one entry, you should return two bytes containing the byte count
required to represent the next entry in the directory.
User code can use this value to formulate a retry if it desires.
See the
DIAGNOSTICS section of
<A href="/magic/man2html/2/stat"><I>stat</I>(2)
</A>for a description of this process.
</P>
<P>
The trickiest part of updating a file server is that the
<TT>clone</TT>
and
<TT>walk</TT>
messages have been merged into a single message, a sort of `clone-multiwalk'.
The new message, still called
<TT>walk</TT>,
proposes a sequence of file name elements to be evaluated using a possibly
cloned fid.
The return message contains the qids of the files reached by
walking to the sequential elements.
If all the elements can be walked, the fid will be cloned if requested.
If a non-zero number of elements are requested, but none
can be walked, an error should be returned.
If only some can be walked, the fid is not cloned, the original fid is left
where it was, and the returned
<TT>Rwalk</TT>
message should contain the partial list of successfully reached qids.
See
<A href="/magic/man2html/5/walk"><I>walk</I>(5)
</A>for a full description.
</P>
<br>&#32;<br>
<A href=http://www.lucent.com/copyright.html>
Copyright</A> &#169; 2004 Lucent Technologies Inc.  All rights reserved.
</body></html>