1
0

il.html 12 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427
  1. <html>
  2. <title>
  3. -
  4. </title>
  5. <body BGCOLOR="#FFFFFF" TEXT="#000000" LINK="#0000FF" VLINK="#330088" ALINK="#FF0044">
  6. <H1>The IL protocol
  7. </H1>
  8. <DL><DD><I>Dave Presotto<br>
  9. Phil Winterbottom<br>
  10. <br>&#32;<br>
  11. presotto,philw@plan9.bell-labs.com<br>
  12. </I></DL>
  13. <DL><DD><H4>ABSTRACT</H4>
  14. To transport the remote procedure call messages of the Plan 9 file system
  15. protocol 9P, we have implemented a new network protocol, called IL.
  16. It is a connection-based, lightweight transport protocol that carries
  17. datagrams encapsulated by IP.
  18. IL provides retransmission of lost messages and in-sequence delivery, but has
  19. no flow control and no blind retransmission.
  20. </DL>
  21. <H4>Introduction
  22. </H4>
  23. <P>
  24. Plan 9 uses a file system protocol, called 9P [PPTTW93], that assumes
  25. in-sequence guaranteed delivery of delimited messages
  26. holding remote procedure call
  27. (RPC) requests and responses.
  28. None of the standard IP protocols [RFC791] is suitable for transmission of
  29. 9P messages over an Ethernet or the Internet.
  30. TCP [RFC793] has a high overhead and does not preserve delimiters.
  31. UDP [RFC768], while cheap and preserving message delimiters, does not provide
  32. reliable sequenced delivery.
  33. When we were implementing IP, TCP, and UDP in our system we
  34. tried to choose a protocol suitable for carrying 9P.
  35. The properties we desired were:
  36. </P>
  37. <DL COMPACT>
  38. <DT>*<DD>
  39. Reliable datagram service
  40. <DT>*<DD>
  41. In-sequence delivery
  42. <DT>*<DD>
  43. Internetworking using IP
  44. <DT>*<DD>
  45. Low complexity, high performance
  46. <DT>*<DD>
  47. Adaptive timeouts
  48. </dl>
  49. <br>&#32;<br>
  50. No standard protocol met our needs so we designed a new one,
  51. called IL (Internet Link).
  52. <P>
  53. IL is a lightweight protocol encapsulated by IP.
  54. It is connection-based and
  55. provides reliable transmission of sequenced messages.
  56. No provision is made for flow control since the protocol
  57. is designed to transport RPC
  58. messages between client and server, a structure with inherent flow limitations.
  59. A small window for outstanding messages prevents too
  60. many incoming messages from being buffered;
  61. messages outside the window are discarded
  62. and must be retransmitted.
  63. Connection setup uses a two-way handshake to generate
  64. initial sequence numbers at each end of the connection;
  65. subsequent data messages increment the
  66. sequence numbers to allow
  67. the receiver to resequence out of order messages.
  68. In contrast to other protocols, IL avoids blind retransmission.
  69. This helps performance in congested networks,
  70. where blind retransmission could cause further
  71. congestion.
  72. Like TCP, IL has adaptive timeouts,
  73. so the protocol performs well both on the
  74. Internet and on local Ethernets.
  75. A round-trip timer is used
  76. to calculate acknowledge and retransmission times
  77. that match the network speed.
  78. </P>
  79. <H4>Connections
  80. </H4>
  81. <P>
  82. An IL connection carries a stream of data between two end points.
  83. While the connection persists,
  84. data entering one side is sent to the other side in the same sequence.
  85. The functioning of a connection is described by the state machine in Figure 1,
  86. which shows the states (circles) and transitions between them (arcs).
  87. Each transition is labeled with the list of events that can cause
  88. the transition and, separated by a horizontal line,
  89. the messages sent or received on that transition.
  90. The remainder of this paper is a discussion of this state machine.
  91. <DL><DT><DD><TT><PRE>
  92. <br><img src="-.15070.gif"><br>
  93. <DL><DD>
  94. </PRE></TT></DL>
  95. </P>
  96. <DL COMPACT>
  97. <DT><I>ackok</I><DD>
  98. any sequence number between id0 and next inclusive
  99. <DT><I>!x</I><DD>
  100. any value except x
  101. <DT>-<DD>
  102. any value
  103. </DL>
  104. <br>&#32;<br>
  105. <I>Figure 1 - IL State Transitions</I>
  106. </dl>
  107. <P>
  108. The IL state machine has five states:
  109. <I>Closed</I>,
  110. <I>Syncer</I>,
  111. <I>Syncee</I>,
  112. <I>Established</I>,
  113. and
  114. <I>Closing</I>.
  115. The connection is identified by the IP address and port number used at each end.
  116. The addresses ride in the IP protocol header, while the ports are part of the
  117. 18-byte IL header.
  118. The local variables identifying the state of a connection are:
  119. <DL><DD>
  120. </P>
  121. <DL COMPACT>
  122. <DT>state<DD>
  123. one of the states
  124. <DT>laddr<DD>
  125. 32-bit local IP address
  126. <DT>lport<DD>
  127. 16-bit local IL port
  128. <DT>raddr<DD>
  129. 32-bit remote IP address
  130. <DT>rport<DD>
  131. 16-bit remote IL port
  132. <DT>id0<DD>
  133. 32-bit starting sequence number of the local side
  134. <DT>rid0<DD>
  135. 32-bit starting sequence number of the remote side
  136. <DT>next<DD>
  137. sequence number of the next message to be sent from the local side
  138. <DT>rcvd<DD>
  139. the last in-sequence message received from the remote side
  140. <DT>unacked<DD>
  141. sequence number of the first unacked message
  142. </DL>
  143. </dl>
  144. <P>
  145. Unused connections are in the
  146. <I>Closed</I>
  147. state with no assigned addresses or ports.
  148. Two events open a connection: the reception of
  149. a message whose addresses and ports match no open connection
  150. or a user explicitly opening a connection.
  151. In the first case, the message's source address and port become the
  152. connection's remote address and port and the message's destination address
  153. and port become the local address and port.
  154. The connection state is set to
  155. <I>Syncee</I>
  156. and the message is processed.
  157. In the second case, the user specifies both local and remote addresses and ports.
  158. The connection's state is set to
  159. <I>Syncer</I>
  160. and a
  161. <TT>sync</TT>
  162. message is sent to the remote side.
  163. The legal values for the local address are constrained by the IP implementation.
  164. </P>
  165. <H4>Sequence Numbers
  166. </H4>
  167. <P>
  168. IL carries data messages.
  169. Each message corresponds to a single write from
  170. the operating system and is identified by a 32-bit
  171. sequence number.
  172. The starting sequence number for each direction in a
  173. connection is picked at random and transmitted in the initial
  174. <TT>sync</TT>
  175. message.
  176. The number is incremented for each subsequent data message.
  177. A retransmitted message contains its original sequence number.
  178. </P>
  179. <H4>Transmission/Retransmission
  180. </H4>
  181. <P>
  182. Each message contains two sequence numbers:
  183. an identifier (ID) and an acknowledgement.
  184. The acknowledgement is the last in-sequence
  185. data message received by the transmitter of the message.
  186. For
  187. <TT>data</TT>
  188. and
  189. <TT>dataquery</TT>
  190. messages, the ID is its sequence number.
  191. For the control messages
  192. <TT>sync</TT>,
  193. <TT>ack</TT>,
  194. <TT>query</TT>,
  195. <TT>state</TT>,
  196. and
  197. <TT>close</TT>,
  198. the ID is one greater than the sequence number of
  199. the highest sent data message.
  200. </P>
  201. <P>
  202. The sender transmits data messages with type
  203. <TT>data</TT>.
  204. Any messages traveling in the opposite direction carry acknowledgements.
  205. An
  206. <TT>ack</TT>
  207. message will be sent within 200 milliseconds of receiving the data message
  208. unless a returning message has already piggy-backed an
  209. acknowledgement to the sender.
  210. </P>
  211. <P>
  212. In IP, messages may be delivered out of order or
  213. may be lost due to congestion or faults.
  214. To overcome this,
  215. IL uses a modified ``go back n'' protocol that also attempts
  216. to avoid aggravating network congestion.
  217. An average round trip time is maintained by measuring the delay between
  218. the transmission of a message and the
  219. receipt of its acknowledgement.
  220. Until the first acknowledge is received, the average round trip time
  221. is assumed to be 100ms.
  222. If an acknowledgement is not received within four round trip times
  223. of the first unacknowledged message
  224. (<I>rexmit timeout</I>
  225. in Figure 1), IL assumes the message or the acknowledgement
  226. has been lost.
  227. The sender then resends only the first unacknowledged message,
  228. setting the type to
  229. <TT>dataquery</TT>.
  230. When the receiver receives a
  231. <TT>dataquery</TT>,
  232. it responds with a
  233. <TT>state</TT>
  234. message acknowledging the highest received in-sequence data message.
  235. This may be the retransmitted message or, if the receiver has been
  236. saving up out-of-sequence messages, some higher numbered message.
  237. Implementations of the receiver are free to choose whether to save out-of-sequence messages.
  238. Our implementation saves up to 10 packets ahead.
  239. When the sender receives the
  240. <TT>state</TT>
  241. message, it will immediately resend the next unacknowledged message
  242. with type
  243. <TT>dataquery</TT>.
  244. This continues until all messages are acknowledged.
  245. </P>
  246. <P>
  247. If no acknowledgement is received after the first
  248. <TT>dataquery</TT>,
  249. the transmitter continues to timeout and resend the
  250. <TT>dataquery</TT>
  251. message.
  252. The intervals between retransmissions increase exponentially.
  253. After 300 times the round trip time
  254. (<I>death timeout</I>
  255. in Figure 1), the sender gives up and
  256. assumes the connection is dead.
  257. </P>
  258. <P>
  259. Retransmission also occurs in the states
  260. <I>Syncer</I>,
  261. <I>Syncee</I>,
  262. and
  263. <I>Close</I>.
  264. The retransmission intervals are the same as for data messages.
  265. </P>
  266. <H4>Keep Alive
  267. </H4>
  268. <P>
  269. Connections to dead systems must be discovered and torn down
  270. lest they consume resources.
  271. If the surviving system does not need to send any data and
  272. all data it has sent has been acknowledged, the protocol
  273. described so far will not discover these connections.
  274. Therefore, in the
  275. <I>Established</I>
  276. state, if no other messages are sent for a 6 second period,
  277. a
  278. <TT>query</TT>
  279. is sent.
  280. The receiver always replies to a
  281. <TT>query</TT>
  282. with a
  283. <TT>state</TT>
  284. message.
  285. If no messages are received for 30 seconds, the
  286. connection is torn down.
  287. This is not shown in Figure 1.
  288. </P>
  289. <H4>Byte Ordering
  290. </H4>
  291. <P>
  292. All 32- and 16-bit quantities are transmitted high-order byte first, as
  293. is the custom in IP.
  294. </P>
  295. <H4>Formats
  296. </H4>
  297. <P>
  298. The following is a C language description of an IP+IL
  299. header, assuming no IP options:
  300. <DL><DT><DD><TT><PRE>
  301. typedef unsigned char byte;
  302. struct IPIL
  303. {
  304. byte vihl; /* Version and header length */
  305. byte tos; /* Type of service */
  306. byte length[2]; /* packet length */
  307. byte id[2]; /* Identification */
  308. byte frag[2]; /* Fragment information */
  309. byte ttl; /* Time to live */
  310. byte proto; /* Protocol */
  311. byte cksum[2]; /* Header checksum */
  312. byte src[4]; /* Ip source */
  313. byte dst[4]; /* Ip destination */
  314. byte ilsum[2]; /* Checksum including header */
  315. byte illen[2]; /* Packet length */
  316. byte iltype; /* Packet type */
  317. byte ilspec; /* Special */
  318. byte ilsrc[2]; /* Src port */
  319. byte ildst[2]; /* Dst port */
  320. byte ilid[4]; /* Sequence id */
  321. byte ilack[4]; /* Acked sequence */
  322. };
  323. </PRE></TT></DL>
  324. </P>
  325. <br>&#32;<br>
  326. Data is assumed to immediately follow the header in the message.
  327. <TT>Ilspec</TT>
  328. is an extension reserved for future protocol changes.
  329. <P>
  330. The checksum is calculated with
  331. <TT>ilsum</TT>
  332. and
  333. <TT>ilspec</TT>
  334. set to zero.
  335. It is the standard IP checksum, that is, the 16-bit one's complement of the one's
  336. complement sum of all 16 bit words in the header and text. If a
  337. message contains an odd number of header and text bytes to be
  338. checksummed, the last byte is padded on the right with zeros to
  339. form a 16-bit word for the checksum.
  340. The checksum covers from
  341. <TT>cksum</TT>
  342. to the end of the data.
  343. </P>
  344. <P>
  345. The possible
  346. <I>iltype</I>
  347. values are:
  348. <DL><DT><DD><TT><PRE>
  349. enum {
  350. sync= 0,
  351. data= 1,
  352. dataquery= 2,
  353. ack= 3,
  354. query= 4,
  355. state= 5,
  356. close= 6,
  357. };
  358. </PRE></TT></DL>
  359. </P>
  360. <br>&#32;<br>
  361. The
  362. <TT>illen</TT>
  363. field is the size in bytes of the IL header (18 bytes) plus the size of the data.
  364. <H4>Numbers
  365. </H4>
  366. <P>
  367. The IP protocol number for IL is 40.
  368. </P>
  369. <P>
  370. The assigned IL port numbers are:
  371. <DL><DD>
  372. </P>
  373. <DL COMPACT>
  374. <DT>7<DD>
  375. echo all input to output
  376. <DT>9<DD>
  377. discard input
  378. <DT>19<DD>
  379. send a standard pattern to output
  380. <DT>565<DD>
  381. send IP addresses of caller and callee to output
  382. <DT>566<DD>
  383. Plan 9 authentication protocol
  384. <DT>17005<DD>
  385. Plan 9 CPU service, data
  386. <DT>17006<DD>
  387. Plan 9 CPU service, notes
  388. <DT>17007<DD>
  389. Plan 9 exported file systems
  390. <DT>17008<DD>
  391. Plan 9 file service
  392. <DT>17009<DD>
  393. Plan 9 remote execution
  394. <DT>17030<DD>
  395. Alef Name Server
  396. </DL>
  397. </dl>
  398. <H4>References
  399. </H4>
  400. <br>&#32;<br>
  401. [PPTTW93] Rob Pike, Dave Presotto, Ken Thompson, Howard Trickey, and Phil Winterbottom,
  402. ``The Use of Name Spaces in Plan 9'',
  403. <I>Op. Sys. Rev.,</I>
  404. Vol. 27, No. 2, April 1993, pp. 72-76,
  405. reprinted in this volume.
  406. <br>
  407. [RFC791] RFC791,
  408. <I>Internet Protocol,</I>
  409. <I>DARPA Internet Program Protocol Specification,</I>
  410. September 1981.
  411. <br>
  412. [RFC793] RFC793,
  413. <I>Transmission Control Protocol,</I>
  414. <I>DARPA Internet Program Protocol Specification,</I>
  415. September 1981.
  416. <br>
  417. [RFC768] J. Postel, RFC768,
  418. <I>User Datagram Protocol,</I>
  419. <I>DARPA Internet Program Protocol Specification,</I>
  420. August 1980.
  421. <br>&#32;<br>
  422. <A href=http://www.lucent.com/copyright.html>
  423. Copyright</A> &#169; 2000 Lucent Technologies Inc. All rights reserved.
  424. </body></html>