prog4.ms 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606
  1. .HTML "Changes to the Programming Environment in the Fourth Release of Plan 9
  2. .FP lucidasans
  3. .TL
  4. Changes to the Programming Environment
  5. .br
  6. in the
  7. .br
  8. Fourth Release of Plan 9
  9. .AU
  10. Rob Pike
  11. .sp
  12. rob@plan9.bell-labs.com
  13. .SH
  14. Introduction
  15. .PP
  16. The fourth release of Plan 9 includes changes at many levels of the system,
  17. with repercussions in the libraries and program interfaces.
  18. This document summarizes the changes and describes how
  19. existing programs must be modified to run in the new release.
  20. It is not exhaustive, of course; for further detail about any of the
  21. topics refer to the manual pages, as always.
  22. .PP
  23. Programmers new to Plan 9 may find valuable tidbits here, but the
  24. real audience for this paper is those with a need to update applications
  25. and servers written in C for earlier releases of the Plan 9 operating system.
  26. .SH
  27. 9P, NAMELEN, and strings
  28. .PP
  29. The underlying file service protocol for Plan 9, 9P, retains its basic form
  30. but has had a number of adjustments to deal with longer file names and error strings,
  31. new authentication mechanisms, and to make it more efficient at
  32. evaluating file names.
  33. The change to file names affects a number of system interfaces;
  34. because file name elements are no longer of fixed size, they can
  35. no longer be stored as arrays.
  36. .PP
  37. 9P used to be a fixed-format protocol with
  38. .CW NAMELEN -sized
  39. byte arrays representing file name elements.
  40. Now, it is a variable-format protocol, as described in
  41. .I intro (5),
  42. in which strings are represented by a count followed by that many bytes.
  43. Thus, the string
  44. .CW ken
  45. would previously have occupied 28
  46. .CW NAMELEN ) (
  47. bytes in the message; now it occupies 5: a two-byte count followed by the three bytes of
  48. .CW ken
  49. and no terminal zero.
  50. (And of course, a name could now be much longer.)
  51. A similar format change has been made to
  52. .CW stat
  53. buffers: they are no longer
  54. .CW DIRLEN
  55. bytes long but instead have variable size prefixed by a two-byte count.
  56. And in fact the entire 9P message syntax has changed: every message
  57. now begins with a message length field that makes it trivial to break the
  58. string into messages without parsing them, so
  59. .CW aux/fcall
  60. is gone.
  61. A new library entry point,
  62. .CW read9pmsg ,
  63. makes it easy for user-level servers to break the client data stream into 9P messages.
  64. All servers should switch from using
  65. .CW read
  66. (or the now gone
  67. .CW getS)
  68. to using
  69. .CW read9pmsg .
  70. .PP
  71. This change to 9P affects the way strings are handled by the kernel and throughout
  72. the system.
  73. The consequences are primarily that fixed-size arrays have been replaced
  74. by pointers and counts in a variety of system interfaces.
  75. Most programs will need at least some adjustment to the new style.
  76. In summary:
  77. .CW NAMELEN
  78. is gone, except as a vestige in the authentication libraries, where it has been
  79. rechristened
  80. .CW ANAMELEN .
  81. .CW DIRLEN
  82. and
  83. .CW ERRLEN
  84. are also gone.
  85. All programs that mention
  86. these constants
  87. will need to be fixed.
  88. .PP
  89. The simplest place to see this change is in the
  90. .CW errstr
  91. system call, which no longer assumes a buffer of length
  92. .CW ERRLEN
  93. but now requires a byte-count argument:
  94. .P1
  95. char buf[...];
  96. errstr(buf, sizeof buf);
  97. .P2
  98. The buffer can be any size you like.
  99. For convenience, the kernel stores error strings internally as 256-byte arrays,
  100. so if you like \(em but it's not required \(em you can use the defined constant
  101. .CW ERRMAX= 256
  102. as a good buffer size.
  103. Unlike the old
  104. .CW ERRLEN
  105. (which had value 64),
  106. .CW ERRMAX
  107. is advisory, not mandatory, and is not part of the 9P specification.
  108. .PP
  109. With names, stat buffers, and directories, there isn't even an echo of a fixed-size array any more.
  110. .SH
  111. Directories and wait messages
  112. .PP
  113. With strings now variable-length, a number of system calls needed to change:
  114. .CW errstr ,
  115. .CW stat ,
  116. .CW fstat ,
  117. .CW wstat ,
  118. .CW fwstat ,
  119. and
  120. .CW wait
  121. are all affected, as is
  122. .CW read
  123. when applied to directories.
  124. .PP
  125. As far as directories are concerned, most programs don't use the system calls
  126. directly anyway, since they operate on the machine-independent form, but
  127. instead call the machine-dependent
  128. .CW Dir
  129. routines
  130. .CW dirstat ,
  131. .CW dirread ,
  132. etc.
  133. These used to fill user-provided fixed-size buffers; now they return objects allocated
  134. by
  135. .CW malloc
  136. (which must therefore be freed after use).
  137. To `stat' a file:
  138. .P1
  139. Dir *d;
  140. d = dirstat(filename);
  141. if(d == nil){
  142. fprint(2, "can't stat %s: %r\en", filename);
  143. exits("stat");
  144. }
  145. use(d);
  146. free(d);
  147. .P2
  148. A common new bug is to forget to free a
  149. .CW Dir
  150. returned by
  151. .CW dirstat .
  152. .PP
  153. .CW Dirfstat
  154. and
  155. .CW Dirfwstat
  156. work pretty much as before, but changes to 9P make
  157. it possible to exercise finer-grained control on what fields
  158. of the
  159. .CW Dir
  160. are to be changed; see
  161. .I stat (2)
  162. and
  163. .I stat (5)
  164. for details.
  165. .PP
  166. Reading a directory works in a similar way to
  167. .CW dirstat ,
  168. with
  169. .CW dirread
  170. allocating and filling in an array of
  171. .CW Dir
  172. structures.
  173. The return value is the number of elements of the array.
  174. The arguments to
  175. .CW dirread
  176. now include a pointer to a
  177. .CW Dir*
  178. to be filled in with the address of the allocated array:
  179. .P1
  180. Dir *d;
  181. int i, n;
  182. while((n = dirread(fd, &d)) > 0){
  183. for(i=0; i<n; i++)
  184. use(&d[i]);
  185. free(d);
  186. }
  187. .P2
  188. A new library function,
  189. .CW dirreadall ,
  190. has the same form as
  191. .CW dirread
  192. but returns the entire directory in one call:
  193. .P1
  194. n = dirreadall(fd, &d)
  195. for(i=0; i<n; i++)
  196. use(&d[i]);
  197. free(d);
  198. .P2
  199. If your program insists on using the underlying
  200. .CW stat
  201. system call or its relatives, or wants to operate directly on the
  202. machine-independent format returned by
  203. .CW stat
  204. or
  205. .CW read ,
  206. it will need to be modified.
  207. Such programs are rare enough that we'll not discuss them here beyond referring to
  208. the man page
  209. .I stat (2)
  210. for details.
  211. Be aware, though, that it used to be possible to regard the buffer returned by
  212. .CW stat
  213. as a byte array that began with the zero-terminated
  214. name of the file; this is no longer true.
  215. With very rare exceptions, programs that call
  216. .CW stat
  217. would be better recast to use the
  218. .CW dir
  219. routines or, if their goal is just to test the existence of a file,
  220. .CW access .
  221. .PP
  222. Similar changes have affected the
  223. .CW wait
  224. system call. In fact,
  225. .CW wait
  226. is no longer a system call but a library routine that calls the new
  227. .CW await
  228. system call and returns a newly allocated machine-dependent
  229. .CW Waitmsg
  230. structure:
  231. .P1
  232. Waitmsg *w;
  233. w = wait();
  234. if(w == nil)
  235. error("wait: %r");
  236. print("pid is %d; exit string %s\en", w->pid, w->msg);
  237. free(w);
  238. .P2
  239. The exit string
  240. .CW w->msg
  241. may be empty but it will never be a nil pointer.
  242. Again, don't forget to free the structure returned by
  243. .CW wait .
  244. If all you need is the pid, you can call
  245. .CW waitpid ,
  246. which reports just the pid and doesn't return an allocated structure:
  247. .P1
  248. int pid;
  249. pid = waitpid();
  250. if(pid < 0)
  251. error("wait: %r");
  252. print("pid is %d\en", pid);
  253. .P2
  254. .SH
  255. Quoted strings and tokenize
  256. .PP
  257. .CW Wait
  258. gives us a good opportunity to describe how the system copes with all this
  259. free-format data.
  260. Consider the text returned by the
  261. .CW await
  262. system call, which includes a set of integers (pids and times) and a string (the exit status).
  263. This information is formatted free-form; here is the statement in the kernel that
  264. generates the message:
  265. .P1
  266. n = snprint(a, n, "%d %lu %lu %lu %q",
  267. wq->w.pid,
  268. wq->w.time[TUser], wq->w.time[TSys], wq->w.time[TReal],
  269. wq->w.msg);
  270. .P2
  271. Note the use of
  272. .CW %q
  273. to produce a quoted-string representation of the exit status.
  274. The
  275. .CW %q
  276. format is like %s but will wrap
  277. .CW rc -style
  278. single quotes around the string if it contains white space or is otherwise ambiguous.
  279. The library routine
  280. .CW tokenize
  281. can be used to parse data formatted this way: it splits white-space-separated
  282. fields but understands the
  283. .CW %q
  284. quoting conventions.
  285. Here is how the
  286. .CW wait
  287. library routine builds its
  288. .CW Waitmsg
  289. from the data returned by
  290. .CW await :
  291. .P1
  292. Waitmsg*
  293. wait(void)
  294. {
  295. int n, l;
  296. char buf[512], *fld[5];
  297. Waitmsg *w;
  298. n = await(buf, sizeof buf-1);
  299. if(n < 0)
  300. return nil;
  301. buf[n] = '\0';
  302. if(tokenize(buf, fld, nelem(fld)) != nelem(fld)){
  303. werrstr("couldn't parse wait message");
  304. return nil;
  305. }
  306. l = strlen(fld[4])+1;
  307. w = malloc(sizeof(Waitmsg)+l);
  308. if(w == nil)
  309. return nil;
  310. w->pid = atoi(fld[0]);
  311. w->time[0] = atoi(fld[1]);
  312. w->time[1] = atoi(fld[2]);
  313. w->time[2] = atoi(fld[3]);
  314. w->msg = (char*)&w[1];
  315. memmove(w->msg, fld[4], l);
  316. return w;
  317. }
  318. .P2
  319. .PP
  320. This style of quoted-string and
  321. .CW tokenize
  322. is used all through the system now.
  323. In particular, devices now
  324. .CW tokenize
  325. the messages written to their
  326. .CW ctl
  327. files, which means that you can send messages that contain white space, by quoting them,
  328. and that you no longer need to worry about whether or not the device accepts a newline.
  329. In other words, you can say
  330. .P1
  331. echo message > /dev/xx/ctl
  332. .P2
  333. instead of
  334. .CW echo
  335. .CW -n
  336. because
  337. .CW tokenize
  338. treats the newline character as white space and discards it.
  339. .PP
  340. While we're on the subject of quotes and strings, note that the implementation of
  341. .CW await
  342. used
  343. .CW snprint
  344. rather than
  345. .CW sprint .
  346. We now deprecate
  347. .CW sprint
  348. because it has no protection against buffer overflow.
  349. We prefer
  350. .CW snprint
  351. or
  352. .CW seprint ,
  353. to constrain the output.
  354. The
  355. .CW %q
  356. format is cleverer than most in this regard:
  357. if the string is too long to be represented in full,
  358. .CW %q
  359. is smart enough to produce a truncated but correctly quoted
  360. string within the available space.
  361. .SH
  362. Mount
  363. .PP
  364. Although strings in 9P are now variable-length and not zero-terminated,
  365. this has little direct effect in most of the system interfaces.
  366. File and user names are still zero-terminated strings as always;
  367. the kernel does the work of translating them as necessary for
  368. transport.
  369. And of course, they are now free to be as long as you might want;
  370. the only hard limit is that their length must be represented in 16 bits.
  371. .PP
  372. One example where this matters is that the file system specification in the
  373. .CW mount
  374. system call can now be much longer.
  375. Programs like
  376. .CW rio
  377. that used the specification string in creative ways were limited by the
  378. .CW NAMELEN
  379. restriction; now they can use the string more freely.
  380. .CW Rio
  381. now accepts a simple but less cryptic specification language for the window
  382. to be created by the
  383. .CW mount
  384. call, e.g.:
  385. .P1
  386. % mount $wsys /mnt/wsys 'new -dx 250 -dy 250 -pid 1234'
  387. .P2
  388. In the old system, this sort of control was impossible through the
  389. .CW mount
  390. interface.
  391. .PP
  392. While we're on the subject of
  393. .CW mount ,
  394. note that with the new security architecture
  395. (see
  396. .I factotum (4)),
  397. 9P has moved its authentication outside the protocol proper.
  398. (For a full description of this change to 9P, see
  399. .I fauth (2),
  400. .I attach (5),
  401. and the paper
  402. .I "Security in Plan 9\f1.)
  403. The most explicit effect of this change is that
  404. .CW mount
  405. now takes another argument,
  406. .CW afd ,
  407. a file descriptor for the
  408. authentication file through which the authentication will be made.
  409. For most user-level file servers, which do not require authentication, it is
  410. sufficient to provide
  411. .CW -1
  412. as the value of
  413. .CW afd:
  414. .P1
  415. if(mount(fd, -1, "/mnt/wsys", MREPL,
  416. "new -dx 250 -dy 250 -pid 1234") < 0)
  417. error("mount failed: %r");
  418. .P2
  419. To connect to servers that require authentication, use the new
  420. .CW fauth
  421. system call or the reimplemented
  422. .CW amount
  423. (authenticated mount) library call.
  424. In fact, since
  425. .CW amount
  426. handles both authenticating and non-authenticating servers, it is often
  427. easiest just to replace calls to
  428. .CW mount
  429. by calls to
  430. .CW amount ;
  431. see
  432. .I auth (2)
  433. for details.
  434. .SH
  435. Print
  436. .PP
  437. The C library has been heavily reworked in places.
  438. Besides the changes mentioned above, it
  439. now has a much more complete set of routines for handling
  440. .CW Rune
  441. strings (that is, zero-terminated arrays of 16-bit character values).
  442. The most sweeping changes, however, are in the way formatted I/O is performed.
  443. .PP
  444. The
  445. .CW print
  446. routine and all its relatives have been reimplemented to offer a number
  447. of improvements:
  448. .IP (1)
  449. Better buffer management, including the provision of an internal flush
  450. routine, makes it unnecessary to provide large buffers.
  451. For example,
  452. .CW print
  453. uses a much smaller buffer now (reducing stack load) while simultaneously
  454. removing the need to truncate the output string if it doesn't fit in the buffer.
  455. .IP (2)
  456. Global variables have been eliminated so no locking is necessary.
  457. .IP (3)
  458. The combination of (1) and (2) means that the standard implementation of
  459. .CW print
  460. now works fine in threaded programs, and
  461. .CW threadprint
  462. is gone.
  463. .IP (4)
  464. The new routine
  465. .CW smprint
  466. prints into, and returns, storage allocated on demand by
  467. .CW malloc .
  468. .IP (5)
  469. It is now possible to print into a
  470. .CW Rune
  471. string; for instance,
  472. .CW runesmprint
  473. is the
  474. .CW Rune
  475. analog of
  476. .CW smprint .
  477. .IP (6)
  478. There is improved support for custom
  479. print verbs and custom output routines such as error handlers.
  480. The routine
  481. .CW doprint
  482. is gone, but
  483. .CW vseprint
  484. can always be used instead.
  485. However, the new routines
  486. .CW fmtfdinit ,
  487. .CW fmtstrinit ,
  488. .CW fmtprint ,
  489. and friends
  490. are often a better replacement.
  491. The details are too long for exposition here;
  492. .I fmtinstall (2)
  493. explains the new interface and provides examples.
  494. .IP (7)
  495. Two new format flags, space and comma, close somewhat the gap between
  496. Plan 9 and ANSI C.
  497. .PP
  498. Despite these changes, most programs will be unaffected;
  499. .CW print
  500. is still
  501. .CW print .
  502. Don't forget, though, that
  503. you should eliminate calls to
  504. .CW sprint
  505. and use the
  506. .CW %q
  507. format when appropriate.
  508. .SH
  509. Binary compatibility
  510. .PP
  511. The discussion so far has been about changes at the source level.
  512. Existing binaries will probably run without change in the new
  513. environment, since the kernel provides backward-compatible
  514. system calls for
  515. .CW errstr ,
  516. .CW stat ,
  517. .CW wait ,
  518. etc.
  519. The only exceptions are programs that do either a
  520. .CW mount
  521. system call, because of the security changes and because
  522. the file descriptor in
  523. .CW mount
  524. must point to a new 9P connection; or a
  525. .CW read
  526. system call on a directory, since the returned data will
  527. be in the new format.
  528. A moment's reflection will discover that this means old
  529. user-level file servers will need to be fixed to run on the new system.
  530. .SH
  531. File servers
  532. .PP
  533. A full description of what user-level servers must do to provide service with
  534. the new 9P is beyond the scope of this paper.
  535. Your best source of information is section 5 of the manual,
  536. combined with study of a few examples.
  537. .CW /sys/src/cmd/ramfs.c
  538. is a simple example; it has a counterpart
  539. .CW /sys/src/lib9p/ramfs.c
  540. that implements the same service using the new
  541. .I 9p (2)
  542. library.
  543. .PP
  544. That said, it's worth summarizing what to watch for when converting a file server.
  545. The
  546. .CW session
  547. message is gone, and there is a now a
  548. .CW version
  549. message that is exchanged at the start of a connection to establish
  550. the version of the protocol to use (there's only one at the moment, identified by
  551. the string
  552. .CW 9P2000 )
  553. and what the maximum message size will be.
  554. This negotiation makes it easier to handle 9P encapsulation, such as with
  555. .CW exportfs ,
  556. and also permits larger message sizes when appropriate.
  557. .PP
  558. If your server wants to authenticate, it will need to implement an authentication file
  559. and implement the
  560. .CW auth
  561. message; otherwise it should return a helpful error string to the
  562. .CW Tauth
  563. request to signal that authentication is not required.
  564. .PP
  565. The handling of
  566. .CW stat
  567. and directory reads will require some changes but they should not be fundamental.
  568. Be aware that seeking on directories is forbidden, so it is fine if you disregard the
  569. file offset when implementing directory reads; this makes it a little easier to handle
  570. the variable-length entries.
  571. You should still never return a partial directory entry; if the I/O count is too small
  572. to return even one entry, you should return two bytes containing the byte count
  573. required to represent the next entry in the directory.
  574. User code can use this value to formulate a retry if it desires.
  575. See the
  576. DIAGNOSTICS section of
  577. .I stat (2)
  578. for a description of this process.
  579. .PP
  580. The trickiest part of updating a file server is that the
  581. .CW clone
  582. and
  583. .CW walk
  584. messages have been merged into a single message, a sort of `clone-multiwalk'.
  585. The new message, still called
  586. .CW walk ,
  587. proposes a sequence of file name elements to be evaluated using a possibly
  588. cloned fid.
  589. The return message contains the qids of the files reached by
  590. walking to the sequential elements.
  591. If all the elements can be walked, the fid will be cloned if requested.
  592. If a non-zero number of elements are requested, but none
  593. can be walked, an error should be returned.
  594. If only some can be walked, the fid is not cloned, the original fid is left
  595. where it was, and the returned
  596. .CW Rwalk
  597. message should contain the partial list of successfully reached qids.
  598. See
  599. .I walk (5)
  600. for a full description.