1
0

prog4.ms 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605
  1. .FP lucidasans
  2. .TL
  3. Changes to the Programming Environment
  4. .br
  5. in the
  6. .br
  7. Fourth Release of Plan 9
  8. .AU
  9. Rob Pike
  10. .sp
  11. rob@plan9.bell-labs.com
  12. .SH
  13. Introduction
  14. .PP
  15. The fourth release of Plan 9 includes changes at many levels of the system,
  16. with repercussions in the libraries and program interfaces.
  17. This document summarizes the changes and describes how
  18. existing programs must be modified to run in the new release.
  19. It is not exhaustive, of course; for further detail about any of the
  20. topics refer to the manual pages, as always.
  21. .PP
  22. Programmers new to Plan 9 may find valuable tidbits here, but the
  23. real audience for this paper is those with a need to update applications
  24. and servers written in C for earlier releases of the Plan 9 operating system.
  25. .SH
  26. 9P, NAMELEN, and strings
  27. .PP
  28. The underlying file service protocol for Plan 9, 9P, retains its basic form
  29. but has had a number of adjustments to deal with longer file names and error strings,
  30. new authentication mechanisms, and to make it more efficient at
  31. evaluating file names.
  32. The change to file names affects a number of system interfaces;
  33. because file name elements are no longer of fixed size, they can
  34. no longer be stored as arrays.
  35. .PP
  36. 9P used to be a fixed-format protocol with
  37. .CW NAMELEN -sized
  38. byte arrays representing file name elements.
  39. Now, it is a variable-format protocol, as described in
  40. .I intro (5),
  41. in which strings are represented by a count followed by that many bytes.
  42. Thus, the string
  43. .CW ken
  44. would previously have occupied 28
  45. .CW NAMELEN ) (
  46. bytes in the message; now it occupies 5: a two-byte count followed by the three bytes of
  47. .CW ken
  48. and no terminal zero.
  49. (And of course, a name could now be much longer.)
  50. A similar format change has been made to
  51. .CW stat
  52. buffers: they are no longer
  53. .CW DIRLEN
  54. bytes long but instead have variable size prefixed by a two-byte count.
  55. And in fact the entire 9P message syntax has changed: every message
  56. now begins with a message length field that makes it trivial to break the
  57. string into messages without parsing them, so
  58. .CW aux/fcall
  59. is gone.
  60. A new library entry point,
  61. .CW read9pmsg ,
  62. makes it easy for user-level servers to break the client data stream into 9P messages.
  63. All servers should switch from using
  64. .CW read
  65. (or the now gone
  66. .CW getS)
  67. to using
  68. .CW read9pmsg .
  69. .PP
  70. This change to 9P affects the way strings are handled by the kernel and throughout
  71. the system.
  72. The consequences are primarily that fixed-size arrays have been replaced
  73. by pointers and counts in a variety of system interfaces.
  74. Most programs will need at least some adjustment to the new style.
  75. In summary:
  76. .CW NAMELEN
  77. is gone, except as a vestige in the authentication libraries, where it has been
  78. rechristened
  79. .CW ANAMELEN .
  80. .CW DIRLEN
  81. and
  82. .CW ERRLEN
  83. are also gone.
  84. All programs that mention
  85. these constants
  86. will need to be fixed.
  87. .PP
  88. The simplest place to see this change is in the
  89. .CW errstr
  90. system call, which no longer assumes a buffer of length
  91. .CW ERRLEN
  92. but now requires a byte-count argument:
  93. .P1
  94. char buf[...];
  95. errstr(buf, sizeof buf);
  96. .P2
  97. The buffer can be any size you like.
  98. For convenience, the kernel stores error strings internally as 256-byte arrays,
  99. so if you like \(em but it's not required \(em you can use the defined constant
  100. .CW ERRMAX= 256
  101. as a good buffer size.
  102. Unlike the old
  103. .CW ERRLEN
  104. (which had value 64),
  105. .CW ERRMAX
  106. is advisory, not mandatory, and is not part of the 9P specification.
  107. .PP
  108. With names, stat buffers, and directories, there isn't even an echo of a fixed-size array any more.
  109. .SH
  110. Directories and wait messages
  111. .PP
  112. With strings now variable-length, a number of system calls needed to change:
  113. .CW errstr ,
  114. .CW stat ,
  115. .CW fstat ,
  116. .CW wstat ,
  117. .CW fwstat ,
  118. and
  119. .CW wait
  120. are all affected, as is
  121. .CW read
  122. when applied to directories.
  123. .PP
  124. As far as directories are concerned, most programs don't use the system calls
  125. directly anyway, since they operate on the machine-independent form, but
  126. instead call the machine-dependent
  127. .CW Dir
  128. routines
  129. .CW dirstat ,
  130. .CW dirread ,
  131. etc.
  132. These used to fill user-provided fixed-size buffers; now they return objects allocated
  133. by
  134. .CW malloc
  135. (which must therefore be freed after use).
  136. To `stat' a file:
  137. .P1
  138. Dir *d;
  139. d = dirstat(filename);
  140. if(d == nil){
  141. fprint(2, "can't stat %s: %r\en", filename);
  142. exits("stat");
  143. }
  144. use(d);
  145. free(d);
  146. .P2
  147. A common new bug is to forget to free a
  148. .CW Dir
  149. returned by
  150. .CW dirstat .
  151. .PP
  152. .CW Dirfstat
  153. and
  154. .CW Dirfwstat
  155. work pretty much as before, but changes to 9P make
  156. it possible to exercise finer-grained control on what fields
  157. of the
  158. .CW Dir
  159. are to be changed; see
  160. .I stat (2)
  161. and
  162. .I stat (5)
  163. for details.
  164. .PP
  165. Reading a directory works in a similar way to
  166. .CW dirstat ,
  167. with
  168. .CW dirread
  169. allocating and filling in an array of
  170. .CW Dir
  171. structures.
  172. The return value is the number of elements of the array.
  173. The arguments to
  174. .CW dirread
  175. now include a pointer to a
  176. .CW Dir*
  177. to be filled in with the address of the allocated array:
  178. .P1
  179. Dir *d;
  180. int i, n;
  181. while((n = dirread(fd, &d)) > 0){
  182. for(i=0; i<n; i++)
  183. use(&d[i]);
  184. free(d);
  185. }
  186. .P2
  187. A new library function,
  188. .CW dirreadall ,
  189. has the same form as
  190. .CW dirread
  191. but returns the entire directory in one call:
  192. .P1
  193. n = dirreadall(fd, &d)
  194. for(i=0; i<n; i++)
  195. use(&d[i]);
  196. free(d);
  197. .P2
  198. If your program insists on using the underlying
  199. .CW stat
  200. system call or its relatives, or wants to operate directly on the
  201. machine-independent format returned by
  202. .CW stat
  203. or
  204. .CW read ,
  205. it will need to be modified.
  206. Such programs are rare enough that we'll not discuss them here beyond referring to
  207. the man page
  208. .I stat (2)
  209. for details.
  210. Be aware, though, that it used to be possible to regard the buffer returned by
  211. .CW stat
  212. as a byte array that began with the zero-terminated
  213. name of the file; this is no longer true.
  214. With very rare exceptions, programs that call
  215. .CW stat
  216. would be better recast to use the
  217. .CW dir
  218. routines or, if their goal is just to test the existence of a file,
  219. .CW access .
  220. .PP
  221. Similar changes have affected the
  222. .CW wait
  223. system call. In fact,
  224. .CW wait
  225. is no longer a system call but a library routine that calls the new
  226. .CW await
  227. system call and returns a newly allocated machine-dependent
  228. .CW Waitmsg
  229. structure:
  230. .P1
  231. Waitmsg *w;
  232. w = wait();
  233. if(w == nil)
  234. error("wait: %r");
  235. print("pid is %d; exit string %s\en", w->pid, w->msg);
  236. free(w);
  237. .P2
  238. The exit string
  239. .CW w->msg
  240. may be empty but it will never be a nil pointer.
  241. Again, don't forget to free the structure returned by
  242. .CW wait .
  243. If all you need is the pid, you can call
  244. .CW waitpid ,
  245. which reports just the pid and doesn't return an allocated structure:
  246. .P1
  247. int pid;
  248. pid = waitpid();
  249. if(pid < 0)
  250. error("wait: %r");
  251. print("pid is %d\en", pid);
  252. .P2
  253. .SH
  254. Quoted strings and tokenize
  255. .PP
  256. .CW Wait
  257. gives us a good opportunity to describe how the system copes with all this
  258. free-format data.
  259. Consider the text returned by the
  260. .CW await
  261. system call, which includes a set of integers (pids and times) and a string (the exit status).
  262. This information is formatted free-form; here is the statement in the kernel that
  263. generates the message:
  264. .P1
  265. n = snprint(a, n, "%d %lud %lud %lud %q",
  266. wq->w.pid,
  267. wq->w.time[TUser], wq->w.time[TSys], wq->w.time[TReal],
  268. wq->w.msg);
  269. .P2
  270. Note the use of
  271. .CW %q
  272. to produce a quoted-string representation of the exit status.
  273. The
  274. .CW %q
  275. format is like %s but will wrap
  276. .CW rc -style
  277. single quotes around the string if it contains white space or is otherwise ambiguous.
  278. The library routine
  279. .CW tokenize
  280. can be used to parse data formatted this way: it splits white-space-separated
  281. fields but understands the
  282. .CW %q
  283. quoting conventions.
  284. Here is how the
  285. .CW wait
  286. library routine builds its
  287. .CW Waitmsg
  288. from the data returned by
  289. .CW await :
  290. .P1
  291. Waitmsg*
  292. wait(void)
  293. {
  294. int n, l;
  295. char buf[512], *fld[5];
  296. Waitmsg *w;
  297. n = await(buf, sizeof buf-1);
  298. if(n < 0)
  299. return nil;
  300. buf[n] = '\0';
  301. if(tokenize(buf, fld, nelem(fld)) != nelem(fld)){
  302. werrstr("couldn't parse wait message");
  303. return nil;
  304. }
  305. l = strlen(fld[4])+1;
  306. w = malloc(sizeof(Waitmsg)+l);
  307. if(w == nil)
  308. return nil;
  309. w->pid = atoi(fld[0]);
  310. w->time[0] = atoi(fld[1]);
  311. w->time[1] = atoi(fld[2]);
  312. w->time[2] = atoi(fld[3]);
  313. w->msg = (char*)&w[1];
  314. memmove(w->msg, fld[4], l);
  315. return w;
  316. }
  317. .P2
  318. .PP
  319. This style of quoted-string and
  320. .CW tokenize
  321. is used all through the system now.
  322. In particular, devices now
  323. .CW tokenize
  324. the messages written to their
  325. .CW ctl
  326. files, which means that you can send messages that contain white space, by quoting them,
  327. and that you no longer need to worry about whether or not the device accepts a newline.
  328. In other words, you can say
  329. .P1
  330. echo message > /dev/xx/ctl
  331. .P2
  332. instead of
  333. .CW echo
  334. .CW -n
  335. because
  336. .CW tokenize
  337. treats the newline character as white space and discards it.
  338. .PP
  339. While we're on the subject of quotes and strings, note that the implementation of
  340. .CW await
  341. used
  342. .CW snprint
  343. rather than
  344. .CW sprint .
  345. We now deprecate
  346. .CW sprint
  347. because it has no protection against buffer overflow.
  348. We prefer
  349. .CW snprint
  350. or
  351. .CW seprint ,
  352. to constrain the output.
  353. The
  354. .CW %q
  355. format is cleverer than most in this regard:
  356. if the string is too long to be represented in full,
  357. .CW %q
  358. is smart enough to produce a truncated but correctly quoted
  359. string within the available space.
  360. .SH
  361. Mount
  362. .PP
  363. Although strings in 9P are now variable-length and not zero-terminated,
  364. this has little direct effect in most of the system interfaces.
  365. File and user names are still zero-terminated strings as always;
  366. the kernel does the work of translating them as necessary for
  367. transport.
  368. And of course, they are now free to be as long as you might want;
  369. the only hard limit is that their length must be represented in 16 bits.
  370. .PP
  371. One example where this matters is that the file system specification in the
  372. .CW mount
  373. system call can now be much longer.
  374. Programs like
  375. .CW rio
  376. that used the specification string in creative ways were limited by the
  377. .CW NAMELEN
  378. restriction; now they can use the string more freely.
  379. .CW Rio
  380. now accepts a simple but less cryptic specification language for the window
  381. to be created by the
  382. .CW mount
  383. call, e.g.:
  384. .P1
  385. % mount $wsys /mnt/wsys 'new -dx 250 -dy 250 -pid 1234'
  386. .P2
  387. In the old system, this sort of control was impossible through the
  388. .CW mount
  389. interface.
  390. .PP
  391. While we're on the subject of
  392. .CW mount ,
  393. note that with the new security architecture
  394. (see
  395. .I factotum (4)),
  396. 9P has moved its authentication outside the protocol proper.
  397. (For a full description of this change to 9P, see
  398. .I fauth (2),
  399. .I attach (5),
  400. and the paper
  401. .I "Security in Plan 9\f1.)
  402. The most explicit effect of this change is that
  403. .CW mount
  404. now takes another argument,
  405. .CW afd ,
  406. a file descriptor for the
  407. authentication file through which the authentication will be made.
  408. For most user-level file servers, which do not require authentication, it is
  409. sufficient to provide
  410. .CW -1
  411. as the value of
  412. .CW afd:
  413. .P1
  414. if(mount(fd, -1, "/mnt/wsys", MREPL,
  415. "new -dx 250 -dy 250 -pid 1234") < 0)
  416. error("mount failed: %r");
  417. .P2
  418. To connect to servers that require authentication, use the new
  419. .CW fauth
  420. system call or the reimplemented
  421. .CW amount
  422. (authenticated mount) library call.
  423. In fact, since
  424. .CW amount
  425. handles both authenticating and non-authenticating servers, it is often
  426. easiest just to replace calls to
  427. .CW mount
  428. by calls to
  429. .CW amount ;
  430. see
  431. .I auth (2)
  432. for details.
  433. .SH
  434. Print
  435. .PP
  436. The C library has been heavily reworked in places.
  437. Besides the changes mentioned above, it
  438. now has a much more complete set of routines for handling
  439. .CW Rune
  440. strings (that is, zero-terminated arrays of 16-bit character values).
  441. The most sweeping changes, however, are in the way formatted I/O is performed.
  442. .PP
  443. The
  444. .CW print
  445. routine and all its relatives have been reimplemented to offer a number
  446. of improvements:
  447. .IP (1)
  448. Better buffer management, including the provision of an internal flush
  449. routine, makes it unnecessary to provide large buffers.
  450. For example,
  451. .CW print
  452. uses a much smaller buffer now (reducing stack load) while simultaneously
  453. removing the need to truncate the output string if it doesn't fit in the buffer.
  454. .IP (2)
  455. Global variables have been eliminated so no locking is necessary.
  456. .IP (3)
  457. The combination of (1) and (2) means that the standard implementation of
  458. .CW print
  459. now works fine in threaded programs, and
  460. .CW threadprint
  461. is gone.
  462. .IP (4)
  463. The new routine
  464. .CW smprint
  465. prints into, and returns, storage allocated on demand by
  466. .CW malloc .
  467. .IP (5)
  468. It is now possible to print into a
  469. .CW Rune
  470. string; for instance,
  471. .CW runesmprint
  472. is the
  473. .CW Rune
  474. analog of
  475. .CW smprint .
  476. .IP (6)
  477. There is improved support for custom
  478. print verbs and custom output routines such as error handlers.
  479. The routine
  480. .CW doprint
  481. is gone, but
  482. .CW vseprint
  483. can always be used instead.
  484. However, the new routines
  485. .CW fmtfdinit ,
  486. .CW fmtstrinit ,
  487. .CW fmtprint ,
  488. and friends
  489. are often a better replacement.
  490. The details are too long for exposition here;
  491. .I fmtinstall (2)
  492. explains the new interface and provides examples.
  493. .IP (7)
  494. Two new format flags, space and comma, close somewhat the gap between
  495. Plan 9 and ANSI C.
  496. .PP
  497. Despite these changes, most programs will be unaffected;
  498. .CW print
  499. is still
  500. .CW print .
  501. Don't forget, though, that
  502. you should eliminate calls to
  503. .CW sprint
  504. and use the
  505. .CW %q
  506. format when appropriate.
  507. .SH
  508. Binary compatibility
  509. .PP
  510. The discussion so far has been about changes at the source level.
  511. Existing binaries will probably run without change in the new
  512. environment, since the kernel provides backward-compatible
  513. system calls for
  514. .CW errstr ,
  515. .CW stat ,
  516. .CW wait ,
  517. etc.
  518. The only exceptions are programs that do either a
  519. .CW mount
  520. system call, because of the security changes and because
  521. the file descriptor in
  522. .CW mount
  523. must point to a new 9P connection; or a
  524. .CW read
  525. system call on a directory, since the returned data will
  526. be in the new format.
  527. A moment's reflection will discover that this means old
  528. user-level file servers will need to be fixed to run on the new system.
  529. .SH
  530. File servers
  531. .PP
  532. A full description of what user-level servers must do to provide service with
  533. the new 9P is beyond the scope of this paper.
  534. Your best source of information is section 5 of the manual,
  535. combined with study of a few examples.
  536. .CW /sys/src/cmd/ramfs.c
  537. is a simple example; it has a counterpart
  538. .CW /sys/src/lib9p/ramfs.c
  539. that implements the same service using the new
  540. .I 9p (2)
  541. library.
  542. .PP
  543. That said, it's worth summarizing what to watch for when converting a file server.
  544. The
  545. .CW session
  546. message is gone, and there is a now a
  547. .CW version
  548. message that is exchanged at the start of a connection to establish
  549. the version of the protocol to use (there's only one at the moment, identified by
  550. the string
  551. .CW 9P2000 )
  552. and what the maximum message size will be.
  553. This negotiation makes it easier to handle 9P encapsulation, such as with
  554. .CW exportfs ,
  555. and also permits larger message sizes when appropriate.
  556. .PP
  557. If your server wants to authenticate, it will need to implement an authentication file
  558. and implement the
  559. .CW auth
  560. message; otherwise it should return a helpful error string to the
  561. .CW Tauth
  562. request to signal that authentication is not required.
  563. .PP
  564. The handling of
  565. .CW stat
  566. and directory reads will require some changes but they should not be fundamental.
  567. Be aware that seeking on directories is forbidden, so it is fine if you disregard the
  568. file offset when implementing directory reads; this makes it a little easier to handle
  569. the variable-length entries.
  570. You should still never return a partial directory entry; if the I/O count is too small
  571. to return even one entry, you should return two bytes containing the byte count
  572. required to represent the next entry in the directory.
  573. User code can use this value to formulate a retry if it desires.
  574. See the
  575. DIAGNOSTICS section of
  576. .I stat (2)
  577. for a description of this process.
  578. .PP
  579. The trickiest part of updating a file server is that the
  580. .CW clone
  581. and
  582. .CW walk
  583. messages have been merged into a single message, a sort of `clone-multiwalk'.
  584. The new message, still called
  585. .CW walk ,
  586. proposes a sequence of file name elements to be evaluated using a possibly
  587. cloned fid.
  588. The return message contains the qids of the files reached by
  589. walking to the sequential elements.
  590. If all the elements can be walked, the fid will be cloned if requested.
  591. If a non-zero number of elements are requested, but none
  592. can be walked, an error should be returned.
  593. If only some can be walked, the fid is not cloned, the original fid is left
  594. where it was, and the returned
  595. .CW Rwalk
  596. message should contain the partial list of successfully reached qids.
  597. See
  598. .I walk (5)
  599. for a full description.