plumb.html 54 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478479480481482483484485486487488489490491492493494495496497498499500501502503504505506507508509510511512513514515516517518519520521522523524525526527528529530531532533534535536537538539540541542543544545546547548549550551552553554555556557558559560561562563564565566567568569570571572573574575576577578579580581582583584585586587588589590591592593594595596597598599600601602603604605606607608609610611612613614615616617618619620621622623624625626627628629630631632633634635636637638639640641642643644645646647648649650651652653654655656657658659660661662663664665666667668669670671672673674675676677678679680681682683684685686687688689690691692693694695696697698699700701702703704705706707708709710711712713714715716717718719720721722723724725726727728729730731732733734735736737738739740741742743744745746747748749750751752753754755756757758759760761762763764765766767768769770771772773774775776777778779780781782783784785786787788789790791792793794795796797798799800801802803804805806807808809810811812813814815816817818819820821822823824825826827828829830831832833834835836837838839840841842843844845846847848849850851852853854855856857858859860861862863864865866867868869870871872873874875876877878879880881882883884885886887888889890891892893894895896897898899900901902903904905906907908909910911912913914915916917918919920921922923924925926927928929930931932933934935936937938939940941942943944945946947948949950951952953954955956957958959960961962963964965966967968969970971972973974975976977978979980981982983984985986987988989990991992993994995996997998999100010011002100310041005100610071008100910101011101210131014101510161017101810191020102110221023102410251026102710281029103010311032103310341035103610371038103910401041104210431044104510461047104810491050105110521053105410551056105710581059106010611062106310641065106610671068106910701071107210731074107510761077107810791080108110821083108410851086108710881089109010911092109310941095109610971098109911001101110211031104110511061107110811091110111111121113111411151116111711181119112011211122112311241125112611271128112911301131113211331134113511361137113811391140114111421143114411451146114711481149115011511152115311541155115611571158115911601161116211631164116511661167116811691170117111721173117411751176117711781179118011811182118311841185118611871188118911901191119211931194119511961197119811991200120112021203120412051206120712081209121012111212121312141215121612171218121912201221122212231224122512261227122812291230123112321233123412351236123712381239124012411242124312441245124612471248124912501251125212531254125512561257125812591260126112621263126412651266126712681269127012711272127312741275127612771278127912801281128212831284128512861287128812891290129112921293129412951296129712981299130013011302130313041305130613071308130913101311131213131314131513161317131813191320132113221323132413251326132713281329133013311332133313341335133613371338133913401341134213431344134513461347134813491350135113521353135413551356135713581359136013611362136313641365136613671368136913701371137213731374137513761377137813791380138113821383138413851386138713881389139013911392139313941395139613971398139914001401140214031404140514061407
  1. <html>
  2. <title>
  3. -
  4. </title>
  5. <body BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000FF" VLINK="#330088" ALINK="#FF0044">
  6. <H1>Plumbing and Other Utilities
  7. </H1>
  8. <DL><DD><I>Rob Pike<br>
  9. Bell Laboratories, Murray Hill, NJ, 07974
  10. </I></DL>
  11. <DL><DD><H4>ABSTRACT</H4>
  12. <br>&#32;<br>
  13. Plumbing is a new mechanism for inter-process communication in Plan 9,
  14. specifically the passing of messages between interactive programs as part of
  15. the user interface.
  16. Although plumbing shares some properties with familiar notions
  17. such as cut and paste,
  18. it offers a more general data exchange mechanism without imposing
  19. a particular user interface.
  20. <br>&#32;<br>
  21. The core of the plumbing system is a program called the
  22. <I>plumber</I>,
  23. which handles all messages and dispatches and reformats them
  24. according to configuration rules written in a special-purpose language.
  25. This approach allows the contents and context of a piece of data to define how
  26. it is handled.
  27. Unlike with drag and drop or cut and paste,
  28. the user doesn't need to deliver the data;
  29. the contents of a plumbing message, as interpreted by the plumbing rules,
  30. determine its destination.
  31. <br>&#32;<br>
  32. The plumber has an unusual architecture: it is a language-driven file server.
  33. This design has distinct advantages.
  34. It makes plumbing easy to add to an existing, Unix-like command environment;
  35. it guarantees uniform handling of inter-application messages;
  36. it off-loads from those applications most of the work of extracting and dispatching messages;
  37. and it works transparently across a network.
  38. </DL>
  39. <H4>Introduction
  40. </H4>
  41. <br>&#32;<br>
  42. Data moves from program to program in myriad ways.
  43. Command-line arguments,
  44. shell pipe lines,
  45. cut and paste,
  46. drag and drop, and other user interface techniques all provide some form
  47. of interprocess communication.
  48. Then there are tricks associated with special domains,
  49. such as HTML hyperlinks or the heuristics mail readers
  50. use to highlight URLs embedded in mail messages.
  51. Some systems provide implicit ways to automate the attachment of program to data&#173;the
  52. best known examples are probably the resource forks in MacOS and the
  53. file name extension `associations' in Microsoft Windows&#173;but in practice
  54. humans must too often carry their data from program to program.
  55. <br>&#32;<br>
  56. Why should a human do the work?
  57. Usually there is one obvious thing to do with a piece of data,
  58. and the data itself suggests what this is.
  59. Resource forks and associations speak to this issue directly, but statically and narrowly and with
  60. little opportunity to control the behavior.
  61. Mechanisms with more generality,
  62. such as cut and paste or drag and drop, demand too much manipulation by
  63. the user and are (therefore) too error-prone.
  64. <br>&#32;<br>
  65. We want a system that, given a piece of data,
  66. hands it to the appropriate application by default with little or no human intervention,
  67. while still permitting the user to override the defaults if desired.
  68. <br>&#32;<br>
  69. The plumbing system is an attempt to address some of these issues in a single,
  70. coherent, central way.
  71. It provides a mechanism for
  72. formatting and sending arbitrary messages between applications,
  73. typically interactive programs such as text editors, web browsers, and the window system,
  74. under the control of a central message-handling server called the
  75. <I>plumber</I>.
  76. Interactive programs provide application-specific connections to the plumber,
  77. triggering with minimal user action the transfer of data or control to other programs.
  78. The result is similar to a hypertext system in which all the links are implicit,
  79. extracted automatically by examining the data and the user's actions.
  80. It obviates
  81. cut and paste and other such hand-driven interprocess communication mechanisms.
  82. Plumbing delivers the goods to the right place automatically.
  83. <H4>Overview
  84. </H4>
  85. <br>&#32;<br>
  86. The plumber is implemented as a Plan 9 file server [Pike93];
  87. programs send messages by writing them to the plumber's file
  88. <TT>/mnt/plumb/send</TT>,
  89. and receive messages by reading them from
  90. <I>ports</I>,
  91. which are other plumber files in
  92. <TT>/mnt/plumb</TT>.
  93. For example,
  94. <TT>/mnt/plumb/edit</TT>
  95. is by convention the file from which a text editor reads messages requesting it to
  96. open and display a file for editing.
  97. (See Figure 1.)
  98. <DL><DT><DD><TT><PRE>
  99. <br><img src="-.14230.gif"><br>
  100. </PRE></TT></DL>
  101. <DL>
  102. <DT><DT>&#32;<DD>
  103. Figure 1. The plumber controls the flow of messages between applications.
  104. Programs write to the file
  105. <TT>send</TT>
  106. and receive on `ports' of various names representing services such as
  107. <TT>edit</TT>
  108. or
  109. <TT>web</TT>.
  110. Although the figure doesn't illustrate it, some programs may both send and receive messages,
  111. and some ports are read by multiple applications.
  112. <br>&#32;<br>
  113. </dl>
  114. <br>&#32;<br>
  115. The plumber takes messages from the
  116. <TT>send</TT>
  117. file and interprets their contents using rules defined by
  118. a special-purpose pattern-action language.
  119. The language specifies any rewriting of the message that is to be done by the plumber
  120. and defines how to dispose of a message, such as by sending it to a port or
  121. starting a new process to handle it.
  122. <br>&#32;<br>
  123. The behavior is best described by example.
  124. Imagine that the user has, in a terminal emulator window,
  125. just run a compilation that has failed:
  126. <DL><DT><DD><TT><PRE>
  127. % make
  128. cc -c rmstar.c
  129. rmstar.c:32: syntax error
  130. ...
  131. </PRE></TT></DL>
  132. The user points the typing cursor somewhere in the string
  133. <TT>rmstar.c:32:</TT>
  134. and executes the
  135. <TT>plumb</TT>
  136. menu entry.
  137. This causes the terminal emulator to format a plumbing message
  138. containing the entire string surrounding the cursor,
  139. <TT>rmstar:32:</TT>,
  140. and to write it to
  141. <TT>/mnt/plumb/send</TT>.
  142. The plumber receives this message and compares it sequentially to the various
  143. patterns in its configuration.
  144. Eventually, it will find one that breaks the string into pieces,
  145. <TT>rmstar.c</TT>,
  146. a colon,
  147. <TT>32</TT>,
  148. and the final colon.
  149. Other associated patterns verify that
  150. <TT>rmstar.c</TT>
  151. is a file in the current directory of the program generating
  152. the message, and that
  153. <TT>32</TT>
  154. looks like a line number within it.
  155. The plumber rewrites the message,
  156. setting the data to the string
  157. <TT>rmstar.c</TT>
  158. and attaching an indication that
  159. <TT>32</TT>
  160. is a line number to display.
  161. Finally, it sends the resulting message to the
  162. <TT>edit</TT>
  163. port.
  164. The text editor picks up the message, opens
  165. <TT>rmstar.c</TT>
  166. (if it's not already open) and highlights line 32, the location of the syntax error.
  167. <br>&#32;<br>
  168. From the user's point of view, this process is simple: the error message appears,
  169. it is `plumbed', and the editor jumps to the problem.
  170. <br>&#32;<br>
  171. Of course, there are many different ways to cause compiler messages to
  172. pop up the source of an error,
  173. but the design of the plumber addresses more general issues than the specific
  174. goal of shortening the compile/debug/edit cycle.
  175. It facilitates the general exchange of data among programs, interactive or otherwise,
  176. throughout the environment, and its
  177. architecture&#173;a central, language-driven file server&#173;although
  178. unusual, has distinct advantages.
  179. It makes plumbing easy to add to an existing, Unix-like command environment;
  180. it guarantees uniform handling of inter-application messages;
  181. it off-loads from those applications most of the work of extracting and dispatching messages;
  182. and it works transparently and effortlessly across a network.
  183. <br>&#32;<br>
  184. This paper is organized bottom-up, beginning with the format of the messages
  185. and proceeding through the plumbing language, the handling of messages,
  186. and the interactive user interface.
  187. The last sections discuss the implications of the design
  188. and compare the plumbing system to other environments that
  189. provide similar services.
  190. <H4>Format of messages
  191. </H4>
  192. <br>&#32;<br>
  193. Since the language that controls the plumber is defined in terms of the
  194. contents of plumbing messages, we begin by describing their layout.
  195. <br>&#32;<br>
  196. Plumbing messages have a fixed-format textual
  197. header followed by a free-format data section.
  198. The header consists of six lines of text, in set order,
  199. each specifying a property of the message.
  200. Any line may be blank except the last, which is the length of the data portion of the
  201. message, as a decimal string.
  202. The lines are, in order:
  203. <DL>
  204. <DT><DT>&#32;<DD>
  205. The source application, the name of the program generating the message.
  206. <DT><DT>&#32;<DD>
  207. The destination port, the name of the port to which the messages should be sent.
  208. <DT><DT>&#32;<DD>
  209. The working directory in which the message was generated.
  210. <DT><DT>&#32;<DD>
  211. The type of the data, analogous to a MIME type, such as
  212. <TT>text</TT>
  213. or
  214. <TT>image/gif</TT>.
  215. <DT><DT>&#32;<DD>
  216. Attributes of the message, given as blank-separated
  217. <I>name<TT>=</TT>value</I>
  218. pairs.
  219. The values may be quoted to protect
  220. blanks or quotes; values may not contain newlines.
  221. <DT><DT>&#32;<DD>
  222. The length of the data section, in bytes.
  223. </dl>
  224. <br>&#32;<br>
  225. Here is a sample message, one that (conventionally) tells the editor to open the file
  226. <TT>/usr/rob/src/mem.c</TT>
  227. and display line
  228. 27 within it:
  229. <DL><DT><DD><TT><PRE>
  230. plumbtest
  231. edit
  232. /usr/rob/src
  233. text
  234. addr=27
  235. 5
  236. mem.c
  237. </PRE></TT></DL>
  238. Because in general it need not be text, the data section of the message has no terminating newline.
  239. <br>&#32;<br>
  240. A library interface simplifies the processing of messages by translating them
  241. to and from a data structure,
  242. <TT>Plumbmsg</TT>,
  243. defined like this:
  244. <DL><DT><DD><TT><PRE>
  245. typedef struct Plumbattr Plumbattr;
  246. typedef struct Plumbmsg Plumbmsg;
  247. struct Plumbmsg
  248. {
  249. char *src; /* source application */
  250. char *dst; /* destination port */
  251. char *wdir; /* working directory */
  252. char *type; /* type of data */
  253. Plumbattr *attr; /* attribute list */
  254. int ndata; /* #bytes of data */
  255. char *data;
  256. };
  257. struct Plumbattr
  258. {
  259. char *name;
  260. char *value;
  261. Plumbattr *next;
  262. };
  263. </PRE></TT></DL>
  264. The library also includes routines to send a message, receive a message,
  265. manipulate the attribute list, and so on.
  266. <H4>The Language
  267. </H4>
  268. <br>&#32;<br>
  269. An instance of the plumber runs for each user on each terminal or workstation.
  270. It
  271. begins by reading its rules from the file
  272. <TT>lib/plumbing</TT>
  273. in the user's home directory,
  274. which in turn may use
  275. <TT>include</TT>
  276. statements to interpolate macro definitions and
  277. rules from standard plumbing rule libraries stored in
  278. <TT>/sys/lib/plumb</TT>.
  279. <br>&#32;<br>
  280. The rules control the processing of messages.
  281. They are written in
  282. a pattern-action language comprising a sequence of blank-line-separated
  283. <I>rule</I>
  284. <I>sets</I>,
  285. each of which contains one or more
  286. <I>patterns</I>
  287. followed by one or more
  288. <I>actions</I>.
  289. Each incoming message is compared against the rule sets in order.
  290. If all the patterns within a rule set succeed,
  291. one of the associated actions is taken and processing completes.
  292. <br>&#32;<br>
  293. The syntax of the language is straightforward.
  294. Each rule (pattern or action) has three components, separated by white space:
  295. an
  296. <I>object</I>,
  297. a
  298. <I>verb</I>,
  299. and optional
  300. <I>arguments</I>.
  301. The object
  302. identifies a part of the message, such as
  303. the source application
  304. (<TT>src</TT>),
  305. or the data
  306. portion of the message
  307. (<TT>data</TT>),
  308. or the rule's own arguments
  309. (<TT>arg</TT>);
  310. or it is the keyword
  311. <TT>plumb</TT>,
  312. which introduces an action.
  313. The verb specifies an operation to perform on the object, such as the word
  314. `<TT>is</TT>'
  315. to require precise equality between the object and the argument, or
  316. `<TT>isdir</TT>'
  317. to require that the object be the name of a directory.
  318. <br>&#32;<br>
  319. For instance, this rule set sends messages containing the names of files
  320. ending in
  321. <TT>.gif</TT>,
  322. <TT>.jpg</TT>,
  323. etc. to a program,
  324. <TT>page</TT>,
  325. to display them; it is analogous to a Windows association rule:
  326. <DL><DT><DD><TT><PRE>
  327. # image files go to page
  328. type is text
  329. data matches '[a-zA-Z0-9_\-./]+'
  330. data matches '([a-zA-Z0-9_\-./]+)\.(jpe?g|gif|bit|tiff|ppm)'
  331. arg isfile $0
  332. plumb to image
  333. plumb client page -wi
  334. </PRE></TT></DL>
  335. (Lines beginning with
  336. <TT>#</TT>
  337. are commentary.)
  338. Consider how this rule handles the following message, annotated down the left column for clarity:
  339. <DL><DT><DD><TT><PRE>
  340. <I>src</I> plumbtest
  341. <I>dst</I>
  342. <I>wdir</I> /usr/rob/pics
  343. <I>type</I> text
  344. <I>attr</I>
  345. <I>ndata</I> 9
  346. <I>data</I> horse.gif
  347. </PRE></TT></DL>
  348. The
  349. <TT>is</TT>
  350. verb specifies a precise match, and the
  351. <TT>type</TT>
  352. field of the message is the string
  353. <TT>text</TT>,
  354. so the first pattern succeeds.
  355. The
  356. <TT>matches</TT>
  357. verb invokes a regular expression pattern match of the object (here
  358. <TT>data</TT>)
  359. against the argument pattern.
  360. Both
  361. <TT>matches</TT>
  362. patterns in this rule set will succeed, and in the process set the variables
  363. <TT>$0</TT>
  364. to the matched string,
  365. <TT>$1</TT>
  366. to the first parenthesized submatch, and so on (analogous to
  367. <TT>&</TT>,
  368. <TT>1</TT>,
  369. etc. in
  370. <TT>ed</TT>'s
  371. regular expressions).
  372. The pattern
  373. <TT>arg</TT>
  374. <TT>isfile</TT>
  375. <TT>$0</TT>
  376. verifies that the named file,
  377. <TT>horse.gif</TT>,
  378. is an actual file in the directory
  379. <TT>/usr/rob/pics</TT>.
  380. If all the patterns succeed, one of the actions will be executed.
  381. <br>&#32;<br>
  382. There are two actions in this rule set.
  383. The
  384. <TT>plumb</TT>
  385. <TT>to</TT>
  386. rule specifies
  387. <TT>image</TT>
  388. as the destination port of the message.
  389. By convention, the plumber mounts its services in the directory
  390. <TT>/mnt/plumb</TT>,
  391. so in this case if the file
  392. <TT>/mnt/plumb/image</TT>
  393. has been opened, the message will be made available to the program reading from it.
  394. Note that the message does not name a port, but the rule set that matches
  395. the message does, and that is sufficient to dispatch the message.
  396. If on the other hand a message matches no rule but has an explicit port mentioned,
  397. that too is sufficient.
  398. <br>&#32;<br>
  399. If no client has opened the
  400. <TT>image</TT>
  401. port,
  402. that is, if the program
  403. <TT>page</TT>
  404. is not already running, the
  405. <TT>plumb</TT>
  406. <TT>client</TT>
  407. action gives the execution script to start the application
  408. and send the message on its way; the
  409. <TT>-wi</TT>
  410. arguments tell
  411. <TT>page</TT>
  412. to create a window and to receive its initial arguments from the plumbing port.
  413. The process by which the plumber starts a program is described in more detail in the next section.
  414. <br>&#32;<br>
  415. It may seem odd that there are two
  416. <TT>matches</TT>
  417. rules in this example.
  418. The reason is related to the way the plumber can use the rules themselves
  419. to refine the
  420. <I>data</I>
  421. in the message, somewhat in the manner of Structural Regular Expressions [Pike87a].
  422. For example, consider what happens if the cursor is at the last character of
  423. <DL><DT><DD><TT><PRE>
  424. % make nightmare&#62;horse.gif
  425. </PRE></TT></DL>
  426. and the user asks to plumb what the cursor is pointing at.
  427. The program creating the plumbing
  428. message&#173;in this case the terminal emulator running the window&#173;can send the
  429. entire white-space-delimited string
  430. <TT>nightmare>horse.gif</TT>
  431. or even the entire line, and the combination of
  432. <TT>matches</TT>
  433. rules can determine that the user was referring to the string
  434. <TT>horse.gif</TT>.
  435. The user could of course select the entire string
  436. <TT>horse.gif</TT>,
  437. but it's more convenient just to point in the general location and let the machine
  438. figure out what should be done.
  439. The process is as follows.
  440. <br>&#32;<br>
  441. The application generating the message adds a special attribute to the message, named
  442. <TT>click</TT>,
  443. whose numerical value is the offset of the cursor&#173;the selection point&#173;within the data string.
  444. This attribute tells the plumber two things:
  445. first, that the regular expressions in
  446. <TT>matches</TT>
  447. rules should be used to identify the relevant data;
  448. and second, approximately where the relevant data lies.
  449. The plumber
  450. will then use the first
  451. <TT>matches</TT>
  452. pattern to identify the longest leftmost match that touches the cursor, which will extract the string
  453. <TT>horse.gif</TT>,
  454. and the second pattern will then verify that that names a picture file.
  455. The rule set succeeds and the data is winnowed to the matching substring
  456. before being sent to its destination.
  457. <br>&#32;<br>
  458. Each
  459. <TT>matches</TT>
  460. pattern within a given rule set must match the same portion of the string, which
  461. guarantees that the rule set fails to match a string for which the
  462. second pattern matches only a portion.
  463. For instance, our example rule set should not execute if the data is the string
  464. <TT>horse.gift</TT>,
  465. and although the first pattern will match
  466. <TT>horse.gift</TT>,
  467. the second will match only
  468. <TT>horse.gif</TT>
  469. and the rule set will fail.
  470. <br>&#32;<br>
  471. The same approach of multiple
  472. <TT>matches</TT>
  473. rules can be used to exclude, for instance, a terminal period from
  474. a file name or URL, so a file name or URL at the end of a sentence is recognized properly.
  475. <br>&#32;<br>
  476. If a
  477. <TT>click</TT>
  478. attribute is not specified, all patterns must match the entire string,
  479. so the user has an option:
  480. he or she may select exactly what data to send,
  481. or may instead indicate where the data is by clicking the selection button on the mouse
  482. and letting the machine locate the URL or image file name within the text.
  483. In other words,
  484. the user can control the contents of the message precisely when required,
  485. but the default, simplest action in the user interface does the right thing most of the time.
  486. <H4>How Messages are Handled in the Plumber
  487. </H4>
  488. <br>&#32;<br>
  489. An application creates a message header, fills in whatever fields it wishes to define,
  490. attaches the data, and writes the result to the file
  491. <TT>send</TT>
  492. in the plumber's service directory,
  493. <TT>/mnt/plumb</TT>.
  494. The plumber receives the message and applies the plumbing rules successively to it.
  495. When a rule set matches, the message is dispatched as indicated by that rule set
  496. and processing continues with the next message.
  497. If no rule set matches the message, the plumber indicates this by returning a write
  498. error to the application, that is, the write to
  499. <TT>/mnt/plumb/send</TT>
  500. fails, with the resulting error string
  501. describing the failure.
  502. (Plan 9 uses strings rather than pre-defined numbers to describe error conditions.)
  503. Thus a program can discover whether a plumbing message has been sent successfully.
  504. <br>&#32;<br>
  505. After a matching rule set has been identified, the plumber applies a series of rewriting
  506. steps to the message. Some rewritings are defined by the rule set; others are implicit.
  507. For example, if the message does not specify a destination port, the outgoing message
  508. will be rewritten to identify it.
  509. If the message does specify the port, the rule set will only match if any
  510. <TT>plumb</TT>
  511. <TT>to</TT>
  512. action in the rule set names the same port.
  513. (If it matches no rule sets, but mentions a port, it will be sent there unmodified.)
  514. <br>&#32;<br>
  515. The rule set may contain actions that explicitly rewrite components of the message.
  516. These may modify the attribute list or replace the data section of the message.
  517. Here is a sample rule set that does both.
  518. It matches strings of the form
  519. <TT>plumb.h</TT>
  520. or
  521. <TT>plumb.h:27</TT>.
  522. If that string identifies a file in the standard C include directory,
  523. <TT>/sys/include</TT>,
  524. perhaps with an optional line number, the outgoing message
  525. is rewritten to contain the full path name and an attribute,
  526. <TT>addr</TT>,
  527. to hold the line number:
  528. <DL><DT><DD><TT><PRE>
  529. # .h files are looked up in /sys/include and passed to edit
  530. type is text
  531. data matches '([a-zA-Z0-9]+\.h)(:([0-9]+))?'
  532. arg isfile /sys/include/$1
  533. data set /sys/include/$1
  534. attr add addr=$3
  535. plumb to edit
  536. </PRE></TT></DL>
  537. The
  538. <TT>data</TT>
  539. <TT>set</TT>
  540. rule replaces the contents of the data, and the
  541. <TT>attr</TT>
  542. <TT>add</TT>
  543. rule adds a new attribute to the message.
  544. The intent of this rule is to permit one to plumb an include file name in a C program
  545. to trigger the opening of that file, perhaps at a specified line, in the text editor.
  546. A variant of this rule, discussed below,
  547. tells the editor how to interpret syntax errors from the compiler,
  548. or the output of
  549. <TT>grep</TT>
  550. <TT>-n</TT>,
  551. both of which use a fixed syntax
  552. <I>file<TT>:</TT>line</I>
  553. to identify a line of source.
  554. <br>&#32;<br>
  555. The Plan 9 text editors interpret the
  556. <TT>addr</TT>
  557. attribute as the definition of which portion of the file to display.
  558. In fact, the real rule includes a richer definition of the address syntax,
  559. so one may plumb strings such as
  560. <TT>plumb.h:/plumbsend</TT>
  561. (using a regular expression after the
  562. <TT>/</TT>)
  563. to pop up the declaration of a function in a C header file.
  564. <br>&#32;<br>
  565. Another form of rewriting is that the plumber may modify the attribute list of
  566. the message to clarify how to handle the message.
  567. The primary example of this involves the treatment of the
  568. <TT>click</TT>
  569. attribute, described in the previous section.
  570. If the message contains a
  571. <TT>click</TT>
  572. attribute and the matching rule set uses it to extract the matching substring from the data,
  573. the plumber
  574. deletes the
  575. <TT>click</TT>
  576. attribute and replaces the data with the matching substring.
  577. <br>&#32;<br>
  578. Once the message is rewritten, the actions of the matching rule set are examined.
  579. If the rule set contains a
  580. <TT>plumb</TT>
  581. <TT>to</TT>
  582. action and the corresponding port is open&#173;that is, if a program is already reading
  583. from that port&#173;the message is delivered to the port.
  584. The application will receive the message and handle it as it sees fit.
  585. If the port is not open, a
  586. <TT>plumb</TT>
  587. <TT>start</TT>
  588. or
  589. <TT>plumb</TT>
  590. <TT>client</TT>
  591. action will start a new program to handle the message.
  592. <br>&#32;<br>
  593. The
  594. <TT>plumb</TT>
  595. <TT>start</TT>
  596. action is the simpler: its argument specifies a command to run
  597. instead of passing on the message; the message is discarded.
  598. Here for instance is a rule that, given the process id (pid) of an existing process,
  599. starts the
  600. <TT>acid</TT>
  601. debugger [Wint94] in a new window to examine that process:
  602. <DL><DT><DD><TT><PRE>
  603. # processes go to acid (assuming strlen(pid) &#62;= 2)
  604. type is text
  605. data matches '[a-zA-Z0-9.:_\-/]+'
  606. data matches '[0-9][0-9]+'
  607. arg isdir /proc/$0
  608. plumb start window acid $0
  609. </PRE></TT></DL>
  610. (Note the use of multiple
  611. <TT>matches</TT>
  612. rules to avoid misfires from strings like
  613. <TT>party.1999</TT>.)
  614. The
  615. <TT>arg</TT>
  616. <TT>isdir</TT>
  617. rule checks that the pid represents a running process (or broken one; Plan 9 does not create
  618. <TT>core</TT>
  619. files but leaves broken processes around for debugging) by checking that the process file
  620. system has a directory for that pid [Kill84].
  621. Using this rule, one may plumb the pid string printed by the
  622. <TT>ps</TT>
  623. command or by the operating system when the program breaks;
  624. the debugger will then start automatically.
  625. <br>&#32;<br>
  626. The other startup action,
  627. <TT>plumb</TT>
  628. <TT>client</TT>,
  629. is used when a program will read messages from the plumbing port.
  630. For example,
  631. text editors can read files specified as command arguments, so one could use a
  632. <TT>plumb</TT>
  633. <TT>start</TT>
  634. rule to begin editing a file.
  635. If, however, the editor will read messages from the
  636. <TT>edit</TT>
  637. plumbing port, letting it read the message
  638. from the port insures that it uses other information in the message,
  639. such as the line number to display.
  640. The
  641. <TT>plumb</TT>
  642. <TT>client</TT>
  643. action is therefore like
  644. <TT>plumb</TT>
  645. <TT>start</TT>,
  646. but keeps the message around for delivery when the application opens the port.
  647. Here is the full rule set to pass a regular file to the text editor:
  648. <DL><DT><DD><TT><PRE>
  649. # existing files, possibly tagged by address, go to editor
  650. type is text
  651. data matches '([.a-zA-Z0-9_/\-]*[a-zA-Z0-9_/\-])('$addr')?'
  652. arg isfile $1
  653. data set $1
  654. attr add addr=$3
  655. plumb to edit
  656. plumb client window $editor
  657. </PRE></TT></DL>
  658. If the editor is already running, the
  659. <TT>plumb</TT>
  660. <TT>to</TT>
  661. rule causes it to receive the message on the port.
  662. If not,
  663. the command
  664. `<TT>window</TT>
  665. <TT>$editor</TT>'
  666. will create a new window (using the Plan 9 program
  667. <TT>window</TT>)
  668. to run the editor, and once that starts it will open the
  669. <TT>edit</TT>
  670. plumbing port as usual and discover this first message already waiting.
  671. <br>&#32;<br>
  672. The variables
  673. <TT>$editor</TT>
  674. and
  675. <TT>$addr</TT>
  676. in this rule set
  677. are macros defined in the plumbing rules file; they specify the name of the user's favorite text editor
  678. and a regular expression
  679. that matches that editor's address syntax, such as line numbers and patterns.
  680. This rule set lives in a library of shared plumbing rules that
  681. users' private rules can build on,
  682. so the rule set needs to be adaptable to different editors and their address syntax.
  683. The macro definitions for Acme and Sam [Pike94,Pike87b] look like this:
  684. <DL><DT><DD><TT><PRE>
  685. editor=acme
  686. # or editor=sam
  687. addrelem='((#?[0-9]+)|(/[A-Za-z0-9_\^]+/?)|[.$])'
  688. addr=:($addrelem([,;+\-]$addrelem)*)
  689. </PRE></TT></DL>
  690. <br>&#32;<br>
  691. Finally, the application reads the message from the appropriate port, such as
  692. <TT>/mnt/plumb/edit</TT>,
  693. unpacks it, and goes to work.
  694. <H4>Message Delivery
  695. </H4>
  696. <br>&#32;<br>
  697. In summary, a message is delivered by writing it to the
  698. <TT>send</TT>
  699. file and having the plumber, perhaps after some rewriting, send it to the destination
  700. port or start a new application to handle it.
  701. If no destination can be found by the plumber, the original write to the
  702. <TT>send</TT>
  703. file will fail, and the application will know the message could not be delivered.
  704. <br>&#32;<br>
  705. If multiple applications are reading from the destination port, each will receive
  706. an identical copy of the message; that is, the plumber implements fan-out.
  707. The number of messages delivered is equal to the number of clients that have
  708. opened the destination port.
  709. The plumber queues the messages and makes sure that each application that opened
  710. the port before the message was written gets exactly one copy.
  711. <br>&#32;<br>
  712. This design minimizes blocking in the sending applications, since the write to the
  713. <TT>send</TT>
  714. file can complete as soon as the message has been queued for the appropriate port.
  715. If the plumber waited for the message to be read by the recipient, the sender could
  716. block unnecessarily.
  717. Unfortunately, this design also means that there is no way for a sender to know when
  718. the message has been handled; in fact, there are cases when
  719. the message will not be delivered at all, such as if the recipient exits while there are
  720. still messages in the queue.
  721. Since the plumber is part of a user interface, and not
  722. an autonomous message delivery system,
  723. the decision was made to give the
  724. non-blocking property priority over reliability of message delivery.
  725. In practice, this tradeoff has worked out well:
  726. applications almost always know when a message has failed to be delivered (the
  727. <TT>write</TT>
  728. fails because no destination could be found),
  729. and those occasions when the sender believes incorrectly that the message has been delivered
  730. are both extremely rare and easily recognized by the user&#173;usually because the recipient
  731. application has exited.
  732. <H4>The Rules File
  733. </H4>
  734. <br>&#32;<br>
  735. The plumber begins execution by reading the user's startup plumbing rules file,
  736. <TT>lib/plumbing</TT>.
  737. Since the plumber is implemented as a file server, it can also present its current rules
  738. as a dynamic file, a design that provides an easily understood way to maintain the rules.
  739. <br>&#32;<br>
  740. The file
  741. <TT>/mnt/plumb/rules</TT>
  742. is the text of the rule set the plumber is currently using,
  743. and it may be edited like a regular file to update those rules.
  744. To clear the rules, truncate that file;
  745. to add a new rule set, append to it:
  746. <DL><DT><DD><TT><PRE>
  747. % echo 'type is text
  748. data is self-destruct
  749. plumb start rm -rf $HOME' &#62;&#62; /mnt/plumb/rules
  750. </PRE></TT></DL>
  751. This rule set will take effect immediately.
  752. If it has a syntax error, the write will fail with an error message from the plumber,
  753. such as `malformed rule' or 'undefined verb'.
  754. <br>&#32;<br>
  755. To restore the plumber to its startup configuration,
  756. <DL><DT><DD><TT><PRE>
  757. % cp /usr/$user/lib/plumbing /mnt/plumb/rules
  758. </PRE></TT></DL>
  759. For more sophisticated changes,
  760. one can of course use a regular text editor to modify
  761. <TT>/mnt/plumb/rules</TT>.
  762. <br>&#32;<br>
  763. This simple way of maintaining an active service could profitably be adopted by other systems.
  764. It avoids the need to reboot, to update registries with special tools, or to send asynchronous signals
  765. to critical programs.
  766. <H4>The User Interface
  767. </H4>
  768. <br>&#32;<br>
  769. One unusual property of the plumbing system is that
  770. the user interface that programs provide to access it can vary considerably, yet
  771. the result is nonetheless a unifying force in the environment.
  772. Shells talk to editors, image viewers, and web browsers; debuggers talk to editors;
  773. editors talk to themselves; and the window system talks to everybody.
  774. <br>&#32;<br>
  775. The plumber grew out of some of the ideas of the Acme editor/window-system/user interface [Pike94],
  776. in particular its `acquisition' feature.
  777. With a three-button mouse, clicking the right button in Acme on a piece of text tells Acme to
  778. get the thing being pointed to.
  779. If it is a file name, open the file;
  780. if it is a directory, open a viewer for its contents;
  781. if a line number, go to that line;
  782. if a regular expression, search for it.
  783. This one-click access to anything describable textually was very powerful but had several
  784. limitations, of which the most important were that Acme's rules for interpreting the
  785. text (that is, the implicit hyperlinks) were hard-wired and inflexible, and
  786. that they only applied to and within Acme itself.
  787. One could not, for example, use Acme's power to open an image file, since Acme is
  788. a text-only system.
  789. <br>&#32;<br>
  790. The plumber addresses these limitations, even with Acme itself:
  791. Acme now uses the plumber to interpret the right button clicks for it.
  792. When the right button is clicked on some text,
  793. Acme constructs a plumbing message much as described above,
  794. using the
  795. <TT>click</TT>
  796. attribute and the white-space-delimited text surrounding the click.
  797. It then writes the message to the plumber; if the write succeeds, all is well.
  798. If not, it falls back to its original, internal rules, which will result in a context search
  799. for the word within the current document.
  800. <br>&#32;<br>
  801. If the message is sent successfully, the recipient is likely to be Acme itself, of course:
  802. the request may be to open a file, for example.
  803. Thus Acme has turned the plumber into an external component of its own operation,
  804. while expanding the possibilities; the operation might be to start an image viewer to
  805. open a picture file, something Acme cannot do itself.
  806. The plumber expands the power of Acme's original user interface.
  807. <br>&#32;<br>
  808. Traditional menu-driven programs such as the text editor Sam [Pike87b] and the default
  809. shell window of the window
  810. system
  811. <TT>8&#189;</TT>
  812. [Pike91] cannot dedicate a mouse button solely to plumbing, but they can certainly
  813. dedicate a menu entry.
  814. The editing menu for such programs now contains an entry,
  815. <TT>plumb</TT>,
  816. that creates a plumbing message using the current selection.
  817. (Acme manages to send a message by clicking on the text with one button;
  818. other programs require a click with the select button and then a menu operation.)
  819. For example, after this happens in a shell window:
  820. <DL><DT><DD><TT><PRE>
  821. % make
  822. cc -c shaney.c
  823. shaney.c:232: i undefined
  824. ...
  825. </PRE></TT></DL>
  826. one can click anywhere on the string
  827. <TT>shaney.c:232</TT>,
  828. execute the
  829. <TT>plumb</TT>
  830. menu entry, and have line 232 appear in the text editor, be it Sam or Acme&#173;whichever has the
  831. <TT>edit</TT>
  832. port open.
  833. (If this were an Acme shell window, it would be sufficient to right-click on the string.)
  834. <br>&#32;<br>
  835. [An interesting side line is how the window system knows what directory the
  836. shell is running in; in other words, what value to place in the
  837. <TT>wdir</TT>
  838. field of the plumb message.
  839. Recall that
  840. <TT>8&#189;</TT>
  841. is, like many Plan 9 programs, a file server.
  842. It now serves a new file,
  843. <TT>/dev/wdir</TT>,
  844. that is private to each window.
  845. Programs, in particular the
  846. Plan 9 shell,
  847. <TT>rc</TT>,
  848. can write that file to inform the window system of its current directory.
  849. When a
  850. <TT>cd</TT>
  851. command is executed in an interactive shell,
  852. <TT>rc</TT>
  853. updates the contents of
  854. <TT>/dev/wdir</TT>
  855. and plumbing can proceed with local file names.]
  856. <br>&#32;<br>
  857. Of course, users can plumb image file names, process ids, URLs, and other items&#173;any string
  858. whose syntax and disposition are defined in the plumbing rules file.
  859. An example of how the pieces fit together is the way Plan 9 now handles mail, particularly
  860. MIME-encoded messages.
  861. <br>&#32;<br>
  862. When a new mail message arrives, the mail receiver process sends a plumbing message to the
  863. <TT>newmail</TT>
  864. port, which notifies any interested process that new mail is here.
  865. The plumbing message contains information about the mail, including
  866. its sender, date, and current location in the file system.
  867. The interested processes include a program,
  868. <TT>faces</TT>,
  869. that gives a graphical display of the mail box using
  870. faces to represent the senders of messages [PiPr85],
  871. as well as interactive mail programs such as the Acme mail viewer [Pike94].
  872. The user can then click on the face that appears, and the
  873. <TT>faces</TT>
  874. program will send another plumbing message, this time to the
  875. <TT>showmail</TT>
  876. port.
  877. Here is the rule for that port:
  878. <DL><DT><DD><TT><PRE>
  879. # faces -&#62; new mail window for message
  880. type is text
  881. data matches '[a-zA-Z0-9_\-./]+'
  882. data matches '/mail/fs/[a-zA-Z0-9/]+/[0-9]+'
  883. plumb to showmail
  884. plumb start window edmail -s $0
  885. </PRE></TT></DL>
  886. If a program, such as the Acme mail reader, is reading that port, it will open a new window
  887. in which to display the message.
  888. If not, the
  889. <TT>plumb</TT>
  890. <TT>start</TT>
  891. rule will create a new window and run
  892. <TT>edmail</TT>,
  893. a conventional mail reading process, to examine it.
  894. Notice how the plumbing connects the components of the interface together the same way
  895. regardless of which components are actually being used to view mail.
  896. <br>&#32;<br>
  897. There is more to the mail story.
  898. Naturally, mail boxes in Plan 9 are treated as little file systems, which are synthesized
  899. on demand by a special-purpose file server that takes a flat mail box file and converts
  900. it into a set of directories, one per message, with component files containing the header,
  901. body, MIME information, and so on.
  902. Multi-part MIME messages are unpacked into multi-level directories, like this:
  903. <DL><DT><DD><TT><PRE>
  904. % ls -l /mail/fs/mbox/25
  905. d-r-xr-xr-x M 20 rob rob 0 Nov 21 13:06 /mail/fs/mbox/25/1
  906. d-r-xr-xr-x M 20 rob rob 0 Nov 21 13:06 /mail/fs/mbox/25/2
  907. --r--r--r-- M 20 rob rob 28678 Nov 21 13:06 /mail/fs/mbox/25/body
  908. --r--r--r-- M 20 rob rob 0 Nov 21 13:06 /mail/fs/mbox/25/cc
  909. ...
  910. % mail
  911. 25 messages
  912. : 25
  913. From: presotto
  914. Date: Sun Nov 21 13:05:51 EST 1999
  915. To: rob
  916. Check this out.
  917. ===&#62; 2/ (image/jpeg) [inline]
  918. /mail/fs/mbox/25/2/fabio.jpg
  919. :
  920. </PRE></TT></DL>
  921. Since the components are all (synthetic) files, the user can plumb the pieces
  922. to view embedded pictures, URLs, and so on.
  923. Note that the mail program can plumb the contents of
  924. <TT>inline</TT>
  925. attachments automatically, without user interaction;
  926. in other words, plumbing lets the mailer handle multimedia data
  927. without itself interpreting it.
  928. <br>&#32;<br>
  929. At a more mundane level, a shell command,
  930. <TT>plumb</TT>,
  931. can be used to send messages:
  932. <DL><DT><DD><TT><PRE>
  933. % cd /usr/rob/src
  934. % plumb mem.c
  935. </PRE></TT></DL>
  936. will send the appropriate message to the
  937. <TT>edit</TT>
  938. port.
  939. A surprising use of the
  940. <TT>plumb</TT>
  941. command is in actions within the plumbing rules file.
  942. In our lab, we commonly receive Microsoft Word documents by mail,
  943. but we do not run Microsoft operating systems on our machines so we cannot
  944. view them without at least rebooting.
  945. Therefore, when a Word document arrives in mail, we could plumb the
  946. <TT>.doc</TT>
  947. file but the text editor could not decode it.
  948. However, we have a program,
  949. <TT>doc2txt</TT>,
  950. that decodes the Word file format to extract and format the embedded text.
  951. The solution is to use
  952. <TT>plumb</TT>
  953. in a
  954. <TT>plumb</TT>
  955. <TT>start</TT>
  956. action to invoke
  957. <TT>doc2txt</TT>
  958. on
  959. <TT>.doc</TT>
  960. files and synthesize a plain text file:
  961. <DL><DT><DD><TT><PRE>
  962. # rule set for microsoft word documents
  963. type is text
  964. data matches '[a-zA-Z0-9_\-./]+'
  965. data matches '([a-zA-Z0-9_\-./]+)\.doc'
  966. arg isfile $0
  967. plumb start doc2txt $data | \
  968. plumb -i -d edit -a action=showdata -a filename=$0
  969. </PRE></TT></DL>
  970. The arguments to
  971. <TT>plumb</TT>
  972. tell it to take standard input as its data rather than the text of the arguments
  973. (<TT>-i</TT>),
  974. define the destination port
  975. (<TT>-d</TT>
  976. <TT>edit</TT>),
  977. and set a conventional attribute so the editor knows to show the message data
  978. itself rather than interpret it as a file name
  979. (<TT>-a</TT>
  980. <TT>action=showdata</TT>)
  981. and provide the original file name
  982. (<TT>-a</TT>
  983. <TT>filename=$0</TT>).
  984. Now when a user plumbs a
  985. <TT>.doc</TT>
  986. file the plumbing rules run a process to extract the text and send it as a
  987. temporary file to the editor for viewing.
  988. It's imperfect, but it's easy and it beats rebooting.
  989. <br>&#32;<br>
  990. Another simple example is a rule that turns man pages into hypertext.
  991. Manual page entries of the form
  992. <TT>plumber(1)</TT>
  993. can be clicked on to pop up a window containing the formatted `man page'.
  994. That man page will in turn contain more such citations, which will also be clickable.
  995. The rule is a little like that for Word documents:
  996. <DL><DT><DD><TT><PRE>
  997. # man index entries are synthesized
  998. type is text
  999. data matches '([a-zA-Z0-9_\-./]+)\(([0-9])\)'
  1000. plumb start man $2 $1 | \
  1001. plumb -i -d edit -a action=showdata -a filename=/man/$1($2)
  1002. </PRE></TT></DL>
  1003. <br>&#32;<br>
  1004. There are many other inventive uses of plumbing.
  1005. One more should give some of the flavor.
  1006. We have a shell script,
  1007. <TT>src</TT>,
  1008. that takes as argument the name of an executable binary file.
  1009. It examines the symbol table of the binary to find the source file
  1010. from which it was compiled.
  1011. Since the Plan 9 compilers place full source path names in the symbol table,
  1012. <TT>src</TT>
  1013. can discover the complete file name.
  1014. That is then passed to
  1015. <TT>plumb</TT>,
  1016. complete with the line number to find the
  1017. symbol
  1018. <TT>main</TT>.
  1019. For example,
  1020. <DL><DT><DD><TT><PRE>
  1021. % src plumb
  1022. </PRE></TT></DL>
  1023. is all it takes to pop up an editor window on the
  1024. <TT>main</TT>
  1025. routine of the
  1026. <TT>plumb</TT>
  1027. command, beginning at line 39 of
  1028. <TT>/sys/src/cmd/plumb/plumb.c</TT>.
  1029. Like most uses of plumbing,
  1030. this is not a breakthrough in functionality, but it is a great convenience.
  1031. <H4>Why This Architecture?
  1032. </H4>
  1033. <br>&#32;<br>
  1034. The design of the plumbing system is peculiar:
  1035. a centralized language-based file server does most of the work,
  1036. while compared to other systems the applications themselves
  1037. contribute relatively little.
  1038. This architecture is deliberate, of course.
  1039. <br>&#32;<br>
  1040. That the plumber's behavior is derived from a linguistic description
  1041. gives the system great flexibility and dynamism&#173;rules can be added
  1042. and changed at will, without rebooting&#173;but the existence of a central library of rules
  1043. ensures that, for most users, the environment behaves in well-established ways.
  1044. <br>&#32;<br>
  1045. That the plumber is a file server is perhaps the most unusual aspect of its design,
  1046. but is also one of the most important.
  1047. Messages are passed by regular I/O operations on files, so no extra technology
  1048. such as remote procedure call or request brokers needs to be provided;
  1049. messages are transmitted by familiar means.
  1050. Almost every service in Plan 9 is a file server, so services can be exported
  1051. trivially using the system's remote file system operations [Pike93].
  1052. The plumber is no exception;
  1053. plumbing messages pass routinely across the network to remote applications without
  1054. any special provision,
  1055. in contrast to some commercial IPC mechanisms that become
  1056. significantly more complex when they involve multiple machines.
  1057. As I write this, my window system is talking to applications running on three
  1058. different machines, but they all share a single instance of the plumber and so
  1059. can interoperate to integrate my environment.
  1060. Plan 9 uses a shared file name space
  1061. to combine multiple networked machines&#173;compute servers,
  1062. file servers, and interactive workstations&#173;into a single
  1063. computing environment; plumbing's design as a file server
  1064. is a natural by-product of, and contributor to, the overall system architecture
  1065. [Pike92].
  1066. <br>&#32;<br>
  1067. The centrality of the plumber is also unusual.
  1068. Other systems tend to let the applications determine where messages will go;
  1069. consider mail readers that recognize and highlight URLs in the messages.
  1070. Why should just the mail readers do this, and why should they just do it for URLs?
  1071. (Acme was guilty of similar crimes.)
  1072. The plumber, by removing such decisions to a central authority,
  1073. guarantees that all applications behave the same and simultaneously
  1074. frees them all from figuring out what's important.
  1075. The ability for the plumber to excerpt useful data from within a message
  1076. is critical to the success of this model.
  1077. <br>&#32;<br>
  1078. The entire system is remarkably small.
  1079. The plumber itself is only about two thousand lines of C code.
  1080. Most applications work fine in a plumbing environment without knowing about it at all;
  1081. some need trivial changes such as to standardize their error output;
  1082. a few need to generate and receive plumbing messages.
  1083. But even to add the ability to send and receive messages in a program such as text editor is short work,
  1084. involving typically a few dozen lines of code.
  1085. Plumbing fits well into the existing environment.
  1086. <br>&#32;<br>
  1087. But plumbing is new and it hasn't been pushed far enough yet.
  1088. Most of the work so far has been with textual messages, although
  1089. the underlying system is capable of handling general data.
  1090. We plan to reimplement some of the existing data movement operations,
  1091. such as cut and paste or drag and drop, to use plumbing as their exchange mechanism.
  1092. Since the plumber is a central message handler, it is an obvious place to store the `clipboard'.
  1093. The clipboard could be built as a special port that holds onto messages rather than
  1094. deleting them after delivery.
  1095. Since the clipboard would then be holding a plumbing
  1096. message rather than plain text, as in the current Plan 9 environment,
  1097. it would become possible to cut and paste arbitrary data without
  1098. providing new mechanism.
  1099. In effect, we would be providing a new user interface to the existing plumbing facilities.
  1100. <br>&#32;<br>
  1101. Another possible extension is the ability to override plumbing operations interactively.
  1102. Originally, the plan was to provide a mechanism, perhaps a pop-up menu, that one could
  1103. use to direct messages, for example to send a PostScript file to the editor rather than the
  1104. PostScript viewer by naming an explicit destination in the message.
  1105. Although this deficiency should one day be addressed, it should be done without
  1106. complicating the interface for invoking the default behavior.
  1107. Meanwhile, in practice the default behavior seems to work very well in practice&#173;as it
  1108. must if plumbing is to be successful&#173;so the lack of
  1109. overrides is not keenly felt.
  1110. <H4>Comparison with Other Systems
  1111. </H4>
  1112. <br>&#32;<br>
  1113. The ideas of the plumbing system grew from an
  1114. attempt to generalize the way Acme acquires files and data.
  1115. Systems further from that lineage also share some properties with plumbing.
  1116. Most, however, require explicit linking or message passing rather than
  1117. plumbing's implicit, context-based pattern matching, and none
  1118. has the plumber's design of a language-based file server.
  1119. <br>&#32;<br>
  1120. Reiss's FIELD system [Reis95] probably comes the closest to providing the facilities of the plumber.
  1121. It has a central message-passing mechanism that connects applications together through
  1122. a combination of a library and a pattern-matching central message dispatcher that handles
  1123. message send and reply.
  1124. The main differences between FIELD's message dispatcher and the plumber are first
  1125. that the plumber is based on a special-purpose language while the FIELD
  1126. system uses an object-oriented library, second that the plumber has no concept
  1127. of a reply to a message, and finally that the FIELD system
  1128. has no concept of port.
  1129. But the key distinction is probably in the level of use.
  1130. In FIELD, the message dispatcher is a critical integrating force of the underlying
  1131. programming environment, handling everything from debugging events to
  1132. changing the working directory of a program.
  1133. Plumbing, by contrast, is intended primarily for integrating the user interface
  1134. of existing tools; it is more modest and very much simpler.
  1135. The central advantage of the plumber is its convenience and dynamism;
  1136. the FIELD system does not share the ease with which
  1137. message dispatch rules can be added or modified.
  1138. <br>&#32;<br>
  1139. The inspiration for Acme was
  1140. the user interface to the object-oriented Oberon system [WiGu92].
  1141. Oberon's user interface interprets mouse clicks on strings such as
  1142. <TT>Obj.meth</TT>
  1143. to invoke calls to the method
  1144. <TT>meth</TT>
  1145. of the object
  1146. <TT>Obj</TT>.
  1147. This was the starting point for Acme's middle-button execution [Pike94],
  1148. but nothing in Oberon is much like Acme's right-button `acquisition',
  1149. which was the starting point for the plumber.
  1150. Oberon's implicit method-based linking is not nearly as general as the pattern-matched
  1151. linking of the plumber, nor does its style of user-triggered method call
  1152. correspond well to the more general idea of inter-application communication
  1153. of plumbing messages.
  1154. <br>&#32;<br>
  1155. Microsoft's OLE interface is another relative.
  1156. It allows one application to
  1157. <I>embed</I>
  1158. its own data within another's,
  1159. for example to place an Excel spreadsheet within a Frame document;
  1160. when Frame needs to format the page, it will start Excel itself, or at least some of its
  1161. DLLs, to format the spreadsheet.
  1162. OLE data can only be understood by the application that created it;
  1163. plumbing messages, by contrast, contain arbitrary data with a rigidly formatted header
  1164. that will be interpreted by the pattern matcher and the destination application.
  1165. The plumber's simplified message format may limit its
  1166. flexibility but makes messages easy and efficient to dispatch and to interpret.
  1167. At least for the cut-and-paste style of exchange OLE encourages,
  1168. plumbing gives up some power in return for simplicity, while avoiding
  1169. the need to invoke a vestigial program (if Excel can be called a vestige) every time
  1170. the pasted data is examined.
  1171. Plumbing is also better suited to
  1172. other styles of data exchange, such as connecting compiler errors to the
  1173. text editor.
  1174. <br>&#32;<br>
  1175. The Hyperbole [Wein] package for Emacs adds hypertext facilities to existing documents.
  1176. It includes explicit links and, like plumbing, a rule-driven way to form implicit links.
  1177. Since Emacs is purely textual, like Acme, Hyperbole does not easily extend to driving
  1178. graphical applications, nor does it provide a general interprocess communication method.
  1179. For instance, although Hyperbole provides some integration for mail applications,
  1180. it cannot provide the glue that allows a click on a face icon in an external program to open a
  1181. mail message within the viewer.
  1182. Moreover, since it is not implemented as a file server,
  1183. Hyperbole does not share the advantages of that architecture.
  1184. <br>&#32;<br>
  1185. Henry's
  1186. <TT>error</TT>
  1187. program in 4BSD echoes a small but common use of plumbing.
  1188. It takes the error messages produced by a compiler and drives a text editor
  1189. through the steps of looking at each one in turn; the notion is to quicken the
  1190. compile/edit/debug cycle.
  1191. Similar results are achieved in EMACS by writing special M-LISP
  1192. macros to parse the error messages from various compilers.
  1193. Although for this particular purpose they may be more convenient than plumbing,
  1194. these are specific solutions to a specific problem and lack plumbing's generality.
  1195. <br>&#32;<br>
  1196. Of course, the resource forks in MacOS and the association rules for
  1197. file name extensions in Windows also provide some of the functionality of
  1198. the plumber, although again without the generality or dynamic nature.
  1199. <br>&#32;<br>
  1200. Closer to home, Ousterhout's Tcl (Tool Command Language) [Oust90]
  1201. was originally designed to embed a little command interpreter
  1202. in each application to control interprocess communication and
  1203. provide a level of integration.
  1204. Plumbing, on the other hand, provides minimal support within
  1205. the application, offloading most of the message handling and all the
  1206. command execution to the central plumber.
  1207. <br>&#32;<br>
  1208. The most obvious relative to plumbing is perhaps the hypertext links of a web browser.
  1209. Plumbing differs by synthesizing
  1210. the links on demand.
  1211. Rather than constructing links within a document as in HTML,
  1212. plumbing uses the context of a button click to derive what it should link to.
  1213. That the rules for this decision can be modified dynamically gives it a more
  1214. fluid feel than a standard web browsing world.
  1215. One possibility for future work is to adapt a web browser to use
  1216. plumbing as its link-following engine, much as Acme used plumbing to offload
  1217. its acquisition rules.
  1218. This would connect the web browser to the existing tools, rather than the
  1219. current trend in most systems of replacing the tools by a browser.
  1220. <br>&#32;<br>
  1221. Each of these prior systems&#173;and there are others, e.g. [Pasa93, Free93]&#173;addresses
  1222. a particular need or subset of the
  1223. issues of system integration.
  1224. Plumbing differs because its particular choices were different.
  1225. It focuses on two key issues:
  1226. centralizing and automating the handling of interprocess communication
  1227. among interactive programs,
  1228. and maximizing the convenience (or minimizing the trouble) for the human user
  1229. of its services.
  1230. Moreover, the plumber's implementation as a file server, with messages
  1231. passed over files it controls,
  1232. permits the architecture to work transparently across a network.
  1233. None of the other systems discussed here integrates distributed systems
  1234. as smoothly as local ones without the addition of significant extra technology.
  1235. <H4>Discussion
  1236. </H4>
  1237. <br>&#32;<br>
  1238. There were a few surprises during the development of plumbing.
  1239. The first version of plumbing was done for the Inferno system [Dorw97a,Dorw97b],
  1240. using its file-to-channel mechanism to mediate the IPC.
  1241. Although it was very simple to build, it encountered difficulties because
  1242. the plumber was too disconnected from its clients; in particular, there was
  1243. no way to discover whether a port was in use.
  1244. When plumbing was implemented afresh for Plan 9, it was provided through a true file server.
  1245. Although this was much more work, it paid off handsomely.
  1246. The plumber now knows whether a port is open, which makes it easy to decide whether
  1247. a new program must be started to handle a message,
  1248. and the ability to edit the rules file dynamically is a major advantage.
  1249. Other advantages arise from the file-server design,
  1250. such as
  1251. the ease of exporting plumbing ports across the network to remote machines
  1252. and the implicit security model a file-based interface provides: no one has
  1253. permission to open my private plumbing files.
  1254. <br>&#32;<br>
  1255. On the other hand, Inferno was an all-new environment and the user interface for plumbing was
  1256. able to be made uniform for all applications.
  1257. This was impractical for Plan 9, so more
  1258. <I>ad hoc</I>
  1259. interfaces had to be provided for that environment.
  1260. Yet even in Plan 9 the advantages of efficient,
  1261. convenient, dynamic interprocess communication outweigh the variability of
  1262. the user interface.
  1263. In fact, it is perhaps a telling point that the system works well for a variety of interfaces;
  1264. the provision of a central, convenient message-passing
  1265. service is a good idea regardless of how the programs use it.
  1266. <br>&#32;<br>
  1267. Plumbing's rule language uses only regular expressions and a few special
  1268. rules such as
  1269. <TT>isfile</TT>
  1270. for matching text.
  1271. There is much more that could be done. For example, in the current system a JPEG
  1272. file can be recognized by a
  1273. <TT>.jpg</TT>
  1274. suffix but not by its contents, since the plumbing language has no facility
  1275. for examining the
  1276. <I>contents</I>
  1277. of files named in its messages.
  1278. To address this issue without adding more special rules requires rethinking
  1279. the language itself.
  1280. Although the current system seems a good balance of complexity
  1281. and functionality,
  1282. perhaps a richer, more general-purpose language would
  1283. permit more exotic applications of the plumbing model.
  1284. <br>&#32;<br>
  1285. In conclusion, plumbing adds an effective, easy-to-use inter-application
  1286. communication mechanism to the Plan 9
  1287. user interface.
  1288. Its unusual design as a language-driven file server makes it easy to add
  1289. context-dependent, dynamically interpreted, general-purpose hyperlinks
  1290. to the desktop, for both existing tools and new ones.
  1291. <H4>Acknowledgements
  1292. </H4>
  1293. <br>&#32;<br>
  1294. Dave Presotto wrote the mail file system and
  1295. <TT>edmail</TT>.
  1296. He, Russ Cox, Sape Mullender, and Cliff Young influenced the design, offered useful suggestions,
  1297. and suffered early versions of the software.
  1298. They also made helpful comments on this paper, as did Dennis Ritchie and Brian Kernighan.
  1299. <H4>References
  1300. </H4>
  1301. <br>&#32;<br>
  1302. [Dorw97a]
  1303. Sean Dorward, Rob Pike, David Leo Presotto, Dennis M. Ritchie,
  1304. Howard W. Trickey, and Philip Winterbottom,
  1305. ``Inferno'',
  1306. <I>Proceedings of the IEEE Compcon 97 Conference</I>,
  1307. San Jose, 1997, pp. 241-244.
  1308. <br>&#32;<br>
  1309. [Dorw97b]
  1310. Sean Dorward, Rob Pike, David Leo Presotto, Dennis M. Ritchie,
  1311. Howard W. Trickey, and Philip Winterbottom,
  1312. ``The Inferno Operating System'',
  1313. <I>Bell Labs Technical Journal</I>,
  1314. <B>2</B>,
  1315. 1, Winter, 1997.
  1316. <br>&#32;<br>
  1317. [Free93]
  1318. FreeBSD,
  1319. Syslog configuration file manual
  1320. <A href="/magic/man2html/0/syslog.conf"><I>syslog.conf</I>(0).
  1321. </A><br>&#32;<br>
  1322. [Kill84]
  1323. T. J. Killian,
  1324. ``Processes as Files'',
  1325. <I>Proceedings of the Summer 1984 USENIX Conference</I>,
  1326. Salt Lake City, 1984, pp. 203-207.
  1327. <br>&#32;<br>
  1328. [Oust90]
  1329. John K. Ousterhout,
  1330. ``Tcl: An Embeddable Command Languages'',
  1331. <I>Proceedings of the Winter 1990 USENIX Conference</I>,
  1332. Washington, 1990, pp. 133-146.
  1333. <br>&#32;<br>
  1334. [Pasa93]
  1335. Vern Paxson and Chris Saltmarsh,
  1336. "Glish: A User-Level Software Bus for Loosely-Coupled Distributed Systems" ,
  1337. <I>Proceedings of the Winter 1993 USENIX Conference</I>,
  1338. San Diego, 1993, pp. 141-155.
  1339. <br>&#32;<br>
  1340. [Pike87a]
  1341. Rob Pike,
  1342. ``Structural Regular Expressions'',
  1343. <I>EUUG Spring 1987 Conference Proceedings</I>,
  1344. Helsinki, May 1987, pp. 21-28.
  1345. <br>&#32;<br>
  1346. [Pike87b]
  1347. Rob Pike,
  1348. ``The Text Editor sam'',
  1349. <I>Software - Practice and Experience</I>,
  1350. <B>17</B>,
  1351. 5, Nov. 1987, pp. 813-845.
  1352. <br>&#32;<br>
  1353. [Pike91]
  1354. Rob Pike,
  1355. ``8&#189;, the Plan 9 Window System'',
  1356. <I>Proceedings of the Summer 1991 USENIX Conference</I>,
  1357. Nashville, 1991, pp. 257-265.
  1358. <br>&#32;<br>
  1359. [Pike93]
  1360. Rob Pike, Dave Presotto, Ken Thompson, Howard Trickey, and Phil Winterbottom,
  1361. ``The Use of Name Spaces in Plan 9'',
  1362. <I>Operating Systems Review</I>,
  1363. <B>27</B>,
  1364. 2, April 1993, pp. 72-76.
  1365. <br>&#32;<br>
  1366. [Pike94]
  1367. Rob Pike,
  1368. ``Acme: A User Interface for Programmers'',
  1369. <I>Proceedings of the Winter 1994 USENIX Conference</I>,
  1370. San Francisco, 1994, pp. 223-234.
  1371. <br>&#32;<br>
  1372. [PiPr85]
  1373. Rob Pike and Dave Presotto,
  1374. ``Face the Nation'',
  1375. <I>Proceedings of the USENIX Summer 1985 Conference</I>,
  1376. Portland, 1985, pg. 81.
  1377. <br>&#32;<br>
  1378. [Reis95]
  1379. Steven P. Reiss,
  1380. <I>The FIELD Programming Environment: A Friendly Integrated Environment for Learning and Development</I>,
  1381. Kluwer, Boston, 1995.
  1382. <br>&#32;<br>
  1383. [Wein]
  1384. Bob Weiner,
  1385. <I>Hyperbole User Manual</I>,
  1386. <TT>http://www.cs.indiana.edu/elisp/hyperbole/hyperbole_1.html</TT>
  1387. <br>&#32;<br>
  1388. [Wint94]
  1389. Philip Winterbottom,
  1390. ``ACID: A Debugger based on a Language'',
  1391. <I>Proceedings of the USENIX Winter Conference</I>,
  1392. San Francisco, CA, 1994.
  1393. <br>&#32;<br>
  1394. [WiGu92]
  1395. Niklaus Wirth and Jurg Gutknecht,
  1396. <I>Project Oberon: The Design of an Operating System and Compilers</I>,
  1397. Addison-Wesley, Reading, 1992.
  1398. <br>&#32;<br>
  1399. <A href=http://www.lucent.com/copyright.html>
  1400. Copyright</A> &#169; 2000 Lucent Technologies Inc. All rights reserved.
  1401. </body></html>