1
0

utf.ms 41 KB

12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061626364656667686970717273747576777879808182838485868788899091929394959697989910010110210310410510610710810911011111211311411511611711811912012112212312412512612712812913013113213313413513613713813914014114214314414514614714814915015115215315415515615715815916016116216316416516616716816917017117217317417517617717817918018118218318418518618718818919019119219319419519619719819920020120220320420520620720820921021121221321421521621721821922022122222322422522622722822923023123223323423523623723823924024124224324424524624724824925025125225325425525625725825926026126226326426526626726826927027127227327427527627727827928028128228328428528628728828929029129229329429529629729829930030130230330430530630730830931031131231331431531631731831932032132232332432532632732832933033133233333433533633733833934034134234334434534634734834935035135235335435535635735835936036136236336436536636736836937037137237337437537637737837938038138238338438538638738838939039139239339439539639739839940040140240340440540640740840941041141241341441541641741841942042142242342442542642742842943043143243343443543643743843944044144244344444544644744844945045145245345445545645745845946046146246346446546646746846947047147247347447547647747847948048148248348448548648748848949049149249349449549649749849950050150250350450550650750850951051151251351451551651751851952052152252352452552652752852953053153253353453553653753853954054154254354454554654754854955055155255355455555655755855956056156256356456556656756856957057157257357457557657757857958058158258358458558658758858959059159259359459559659759859960060160260360460560660760860961061161261361461561661761861962062162262362462562662762862963063163263363463563663763863964064164264364464564664764864965065165265365465565665765865966066166266366466566666766866967067167267367467567667767867968068168268368468568668768868969069169269369469569669769869970070170270370470570670770870971071171271371471571671771871972072172272372472572672772872973073173273373473573673773873974074174274374474574674774874975075175275375475575675775875976076176276376476576676776876977077177277377477577677777877978078178278378478578678778878979079179279379479579679779879980080180280380480580680780880981081181281381481581681781881982082182282382482582682782882983083183283383483583683783883984084184284384484584684784884985085185285385485585685785885986086186286386486586686786886987087187287387487587687787887988088188288388488588688788888989089189289389489589689789889990090190290390490590690790890991091191291391491591691791891992092192292392492592692792892993093193293393493593693793893994094194294394494594694794894995095195295395495595695795895996096196296396496596696796896997097197297397497597697797897998098198298398498598698798898999099199299399499599699799899910001001100210031004100510061007100810091010101110121013101410151016101710181019102010211022102310241025102610271028102910301031103210331034103510361037103810391040104110421043104410451046104710481049105010511052105310541055105610571058105910601061106210631064106510661067106810691070107110721073107410751076107710781079108010811082108310841085108610871088108910901091109210931094109510961097109810991100110111021103110411051106110711081109111011111112111311141115111611171118111911201121112211231124112511261127112811291130113111321133113411351136113711381139114011411142114311441145114611471148114911501151115211531154115511561157115811591160116111621163116411651166116711681169117011711172117311741175117611771178117911801181118211831184118511861187118811891190119111921193119411951196119711981199120012011202120312041205120612071208120912101211121212131214121512161217121812191220122112221223122412251226122712281229123012311232123312341235123612371238123912401241124212431244124512461247
  1. .TL
  2. Hello World
  3. .br
  4. or
  5. .br
  6. .ft R
  7. Καλημέρα κόσμε
  8. .ft
  9. .br
  10. or
  11. .br
  12. \f(Jpこんにちは 世界\fP
  13. .AU
  14. Rob Pike
  15. Ken Thompson
  16. .sp
  17. rob,ken@plan9.bell-labs.com
  18. .AB
  19. .FS
  20. Originally appeared, in a slightly different form, in
  21. .I
  22. Proc. of the Winter 1993 USENIX Conf.,
  23. .R
  24. pp. 43-50,
  25. San Diego
  26. .FE
  27. Plan 9 from Bell Labs has recently been converted from ASCII
  28. to an ASCII-compatible variant of the Unicode Standard, a 16-bit character set.
  29. In this paper we explain the reasons for the change,
  30. describe the character set and representation we chose,
  31. and present the programming models and software changes
  32. that support the new text format.
  33. Although we stopped short of full internationalization\(emfor
  34. example, system error messages are in Unixese, not Japanese\(emwe
  35. believe Plan 9 is the first system to treat the representation
  36. of all major languages on a uniform, equal footing throughout all its
  37. software.
  38. .AE
  39. .SH
  40. Introduction
  41. .PP
  42. The world is multilingual but most computer systems
  43. are based on English and ASCII.
  44. The first release of Plan 9 [Pike90], a new distributed operating
  45. system from Bell Laboratories, seemed a good occasion
  46. to correct this chauvinism.
  47. It is easier to make such deep changes when building new systems than
  48. by refitting old ones.
  49. .PP
  50. The ANSI C standard [ANSIC] contains some guidance on the matter of
  51. `wide' and `multi-byte' characters but falls far short of
  52. solving the myriad associated problems.
  53. We could find no literature on how to convert a
  54. .I system
  55. to larger character sets, although some individual
  56. .I programs
  57. had been converted.
  58. This paper reports what we discovered as we
  59. explored the problem of representing multilingual
  60. text at all levels of an operating system,
  61. from the file system and kernel through
  62. the applications and up to the window system
  63. and display.
  64. .PP
  65. Plan 9 has not been `internationalized':
  66. its manuals are in English,
  67. its error messages are in English,
  68. and it can display text that goes from left to right only.
  69. But before we can address these other problems,
  70. we need to handle, uniformly and comfortably,
  71. the textual representation of all the major written languages.
  72. That subproblem is richer than we had anticipated.
  73. .SH
  74. Standards
  75. .PP
  76. Our first step was to select a standard.
  77. At the time (January 1992),
  78. there were only two viable options:
  79. ISO 10646 [ISO10646] and Unicode [Unicode].
  80. The documents describing both proposals were still in the draft stage.
  81. .PP
  82. The draft of ISO 10646 was not
  83. very attractive to us.
  84. It defined a sparse set of 32-bit characters,
  85. which would be
  86. hard to implement
  87. and have punitive storage requirements.
  88. Also, the draft attempted to
  89. mollify national interests by allocating
  90. 16-bit subspaces to national committees
  91. to partition individually.
  92. The suggested mode of use was to
  93. ``flip'' between separate national
  94. standards to implement the international standard.
  95. This did not strike us as a sound basis for a character set.
  96. As well, transmitting 32-bit values in a byte stream,
  97. such as in pipes, would be expensive and hard to implement.
  98. Since the standard does not define a byte order for such
  99. transmission, the byte stream would also have to carry
  100. state to enable the values to be recovered.
  101. .PP
  102. The Unicode Standard is a proposal by a consortium of mostly American
  103. computer companies formed
  104. to protest the technical
  105. failings of ISO 10646.
  106. It defines a uniform 16-bit code based on the
  107. principle of unification:
  108. two characters are the same if they look the
  109. same even though they are from different
  110. languages.
  111. This principle, called Han unification,
  112. allows the large Japanese, Chinese, and Korean
  113. character sets to be packed comfortably into a 16-bit representation.
  114. .PP
  115. We chose the Unicode Standard for its technical merits and because its
  116. code space was better defined.
  117. Moreover,
  118. the Unicode Consortium was derailing the
  119. ISO 10646 standard.
  120. (Now, in 1995,
  121. ISO 10646 is a standard
  122. with one 16-bit group defined,
  123. which is almost exactly the Unicode Standard.
  124. As most people expected, the two standards bodies
  125. reached a détente and
  126. ISO 10646 and Unicode represent the same character set.)
  127. .PP
  128. The Unicode Standard defines an adequate character set
  129. but an unreasonable representation.
  130. It states that all characters
  131. are 16 bits wide and are communicated and stored in
  132. 16-bit units.
  133. It also reserves a pair of characters
  134. (hexadecimal FFFE and FEFF) to detect byte order
  135. in transmitted text, requiring state in the byte stream.
  136. (The Unicode Consortium was thinking of files, not pipes.)
  137. To adopt this encoding,
  138. we would have had to convert all text going
  139. into and out of Plan 9 between ASCII and Unicode, which cannot be done.
  140. Within a single program, in command of all its input and output,
  141. it is possible to define characters as 16-bit quantities;
  142. in the context of a networked system with
  143. hundreds of applications on diverse machines
  144. by different manufacturers,
  145. it is impossible.
  146. .PP
  147. We needed a way to adapt the Unicode Standard to the tools-and-pipes
  148. model of text processing embodied by the Unix system.
  149. To do that, we
  150. needed an ASCII-compatible textual
  151. representation of Unicode characters for transmission
  152. and storage.
  153. In the draft ISO standard there was an informative
  154. (non-required)
  155. Annex
  156. called UTF
  157. that provided a byte stream encoding
  158. of the 32-bit ISO code.
  159. The encoding uses multibyte sequences composed
  160. from the 190 printable characters of Latin-1
  161. to represent character values larger
  162. than 159.
  163. .PP
  164. The UTF encoding has several good properties.
  165. By far the most important is that
  166. a byte in the ASCII range 0-127 represents
  167. itself in UTF.
  168. Thus UTF is backward compatible with ASCII.
  169. .PP
  170. UTF has other advantages.
  171. It is a byte encoding and is
  172. therefore byte-order independent.
  173. ASCII control characters appear in the byte stream
  174. only as themselves, never as an element of a sequence
  175. encoding another character,
  176. so newline bytes separate lines of UTF text.
  177. Finally, ANSI C's
  178. .CW strcmp
  179. function applied to UTF strings preserves the ordering of Unicode characters.
  180. .PP
  181. To encode and decode UTF is expensive (involving multiplication,
  182. division, and modulo operations) but workable.
  183. UTF's major disadvantage is that the encoding
  184. is not self-synchronizing.
  185. It is in general impossible to find the character
  186. boundaries in a UTF string without reading from
  187. the beginning of the string, although in practice
  188. control characters such as newlines,
  189. tabs, and blanks provide synchronization points.
  190. .PP
  191. In August 1992,
  192. X-Open circulated a proposal for another UTF-like
  193. byte encoding of Unicode characters.
  194. Their major concern was that an embedded character
  195. in a file name
  196. (in particular a slash)
  197. could be part of an escape sequence in UTF and
  198. therefore confuse a traditional file system.
  199. Their proposal would allow all 7-bit ASCII characters
  200. to represent themselves
  201. .I "and only themselves"
  202. in text.
  203. Multibyte sequences would contain only characters
  204. with the high bit set.
  205. We proposed a modification to the new UTF that
  206. would address our synchronization problem.
  207. Our proposal, which was originally known informally as UTF-2 and FSS-UTF,
  208. is now referred to as UTF-8 and has been approved by ISO to become
  209. Annex P to ISO 10646.
  210. .PP
  211. The model for text in Plan 9 is chosen from these
  212. three standards*:
  213. .FS
  214. * ``That's the nice thing about standards\(emthere's so many to choose from.'' \- Andy Tannenbaum (no, the other one)
  215. .FE
  216. the Unicode character set encoded as a byte stream by
  217. UTF-8, from
  218. (soon to be) Annex P of ISO 10646.
  219. Although this mixture may seem like a precarious position for us to adopt,
  220. it is not as bad as it sounds.
  221. ISO 10646 and the Unicode Standard have converged,
  222. other systems such as Linux have adopted the same character set and encoding,
  223. and the general feeling seems to be that Unicode and UTF-8 will be accepted as the way
  224. to exchange text between systems.
  225. The prognosis for wide acceptance is good.
  226. .PP
  227. There are a couple of aspects of the Unicode Standard we have not faced.
  228. One is the issue of right-to-left text such as Hebrew or Arabic.
  229. Since that is an issue of display, not representation, we believe
  230. we can defer that problem for the moment without affecting our
  231. ability to solve it later.
  232. Another issue is diacriticals and `combining characters',
  233. which cause overstriking of multiple Unicode characters.
  234. Although necessary for some scripts, such as Thai, Arabic, and Hebrew,
  235. such characters confuse the issues for Latin languages because they
  236. generate multiple representations for accented characters.
  237. ISO 10646 describes three levels of implementation;
  238. in Plan 9 we decided not to address the issue.
  239. Again, this can be labeled as a display issue and its finer points are still being debated,
  240. so we felt comfortable deferring. Mañana.
  241. .PP
  242. Although we converted Plan 9 in the altruistic interests of
  243. serving foreign languages, we have found the large character
  244. set attractive for other reasons. The Unicode Standard includes many
  245. characters\(emmathematical symbols, scientific notation,
  246. more general punctuation, and more\(emthat we now use
  247. daily in our work. We no longer test our imaginations
  248. to find ways to include non-ASCII symbols in our text;
  249. why type
  250. .CW :-)
  251. when you can use the character ☺?
  252. Most compelling is the ability to absorb documents
  253. and data that contain non-ASCII characters; our browser for the
  254. Oxford English Dictionary
  255. lets us see the dictionary as it really is, with pronunciation
  256. in the IPA font, foreign phrases properly rendered, and so on,
  257. .I "in plain text.
  258. .PP
  259. In the rest of this paper, except when
  260. stated otherwise, the term `UTF' refers to the UTF-8 encoding
  261. of Unicode characters as adopted by Plan 9.
  262. .SH
  263. C Compiler
  264. .PP
  265. The first program to be converted to UTF
  266. was the C Compiler.
  267. There are two levels of conversion.
  268. On the syntactic level,
  269. input to the C compiler
  270. is UTF; on the semantic level,
  271. the C language needs to define
  272. how compiled programs manipulate
  273. the UTF set.
  274. .PP
  275. The syntactic part is simple.
  276. The ANSI C language standard defines the
  277. source character set to be ASCII.
  278. Since UTF is backward compatible with ASCII,
  279. the compiler needs little change.
  280. The only places where a larger character set
  281. is allowed are in character constants, strings, and comments.
  282. Since 7-bit ASCII characters can represent only
  283. themselves in UTF,
  284. the compiler does not have to be careful while looking
  285. for the termination of a string or comment.
  286. .PP
  287. The Plan 9 compiler extends ANSI C to treat any Unicode
  288. character with a value outside of the ASCII range as
  289. an alphabetic.
  290. To a Greek programmer or an English mathematician,
  291. α is a sensible and now valid variable name.
  292. .PP
  293. On the semantic level, ANSI C allows,
  294. but does not tie down,
  295. the notion of a
  296. .I "wide character
  297. and admits string and character constants
  298. of this type.
  299. We chose the wide character type to be
  300. .CW unsigned
  301. .CW short .
  302. In the libraries, the word
  303. .CW Rune
  304. is defined by a
  305. .CW typedef
  306. to be equivalent to
  307. .CW unsigned
  308. .CW short
  309. and is
  310. used to signify a Unicode character.
  311. .PP
  312. There are surprises; for example:
  313. .P1
  314. L'x' \f1is 120\fP
  315. \&'x' \f1is 120\fP
  316. L'ÿ' \f1is 255\fP
  317. \&'ÿ' \f1is -1, stdio \fPEOF\f1 (if \fPchar\f1 is signed)\fP
  318. L'\f1α\fP' \f1is 945\fP
  319. \&'\f1α\fP' \f1is illegal\fP
  320. .P2
  321. In the string constants,
  322. .P1
  323. "\f(Jpこんにちは 世界\fP"
  324. L"\f(Jpこんにちは 世界\fP",
  325. .P2
  326. the former is an array of
  327. .CW chars
  328. with 22 elements
  329. and a null byte,
  330. while the latter is an array of
  331. .CW unsigned
  332. .CW shorts
  333. .CW Runes ) (
  334. with 8 elements and a null
  335. .CW Rune .
  336. .PP
  337. The Plan 9 library provides an output conversion function,
  338. .CW print
  339. (analogous to
  340. .CW printf ),
  341. with formats
  342. .CW %c ,
  343. .CW %C ,
  344. .CW %s ,
  345. and
  346. .CW %S .
  347. Since
  348. .CW print
  349. produces text, its output is always UTF.
  350. The character conversion
  351. .CW %c
  352. (lower case) masks its argument
  353. to 8 bits before converting to UTF.
  354. Thus
  355. .CW L'ÿ'
  356. and
  357. .CW 'ÿ'
  358. printed under
  359. .CW %c
  360. will be identical,
  361. but
  362. .CW L'\f1α\fP'
  363. will print as the Unicode
  364. character with decimal value 177.
  365. The character conversion
  366. .CW %C
  367. (upper case) masks its argument
  368. to 16 bits before converting to UTF.
  369. Thus
  370. .CW L'ÿ'
  371. and
  372. .CW L'\f1α\fP'
  373. will print correctly under
  374. .CW %C ,
  375. but
  376. .CW 'ÿ'
  377. will not.
  378. The conversion
  379. .CW %s
  380. (lower case)
  381. expects a pointer to
  382. .CW char
  383. and copies UTF sequences up to a null byte.
  384. The conversion
  385. .CW %S
  386. (upper case) expects a pointer to
  387. .CW Rune
  388. and
  389. performs sequential
  390. .CW %C
  391. conversions until a null
  392. .CW Rune
  393. is encountered.
  394. .PP
  395. Another problem in format conversion
  396. is the definition of
  397. .CW %10s :
  398. does the number refer to bytes or characters?
  399. We decided that such formats were most
  400. often used to align output columns and
  401. so made the number count characters.
  402. Some programs, however, use the count
  403. to place blank-padded strings
  404. in fixed-sized arrays.
  405. These programs must be found and corrected.
  406. .PP
  407. Here is a complete example:
  408. .P1
  409. #include <u.h>
  410. char c[] = "\f(Jpこんにちは 世界\fP";
  411. Rune s[] = L"\f(Jpこんにちは 世界\fP";
  412. main(void)
  413. {
  414. print("%d, %d\en", sizeof(c), sizeof(s));
  415. print("%s\en", c);
  416. print("%S\en", s);
  417. }
  418. .P2
  419. .PP
  420. This program prints
  421. .CW 23,
  422. .CW 18
  423. and then two identical lines of
  424. UTF text.
  425. In practice,
  426. .CW %S
  427. and
  428. .CW L"..."
  429. are rare in programs; one reason is
  430. that most formatted I/O is done in unconverted UTF.
  431. .SH
  432. Ramifications
  433. .PP
  434. All programs in Plan 9 now read and write text as UTF, not ASCII.
  435. This change breaks two deep-rooted symmetries implicit in most C programs:
  436. .IP 1.
  437. A character is no longer a
  438. .CW char .
  439. .IP 2.
  440. The internal representation (Rune) of a character now differs from its
  441. external representation (UTF).
  442. .PP
  443. In the sections that follow,
  444. we show how these issues were faced in the layers of
  445. system software from the operating system up to the applications.
  446. The effects are wide-reaching and often surprising.
  447. .SH
  448. Operating system
  449. .PP
  450. Since UTF is the only format for text in Plan 9,
  451. the interface to the operating system had to be converted to UTF.
  452. Text strings cross the interface in several places:
  453. command arguments,
  454. file names,
  455. user names (people can log in using their native name),
  456. error messages,
  457. and miscellaneous minor places such as commands to the I/O system.
  458. Little change was required: null-terminated UTF strings
  459. are equivalent to null-terminated ASCII strings for most purposes
  460. of the operating system.
  461. The library routines described in the next section made that
  462. change straightforward.
  463. .PP
  464. The window system, once called
  465. .CW 8.5 ,
  466. is now rightfully called
  467. .CW 8½ .
  468. .SH
  469. Libraries
  470. .PP
  471. A header file included by all programs (see [Pike92]) declares
  472. the
  473. .CW Rune
  474. type to hold 16-bit character values:
  475. .P1
  476. typedef unsigned short Rune;
  477. .P2
  478. Also defined are several constants relevant to UTF:
  479. .P1
  480. enum
  481. {
  482. UTFmax = 3, /* maximum bytes per rune */
  483. Runesync = 0x80, /* can't appear in UTF sequence (<) */
  484. Runeself = 0x80, /* rune==UTF sequence (<) */
  485. Runeerror = 0x80, /* decoding error in UTF */
  486. };
  487. .P2
  488. (With the original UTF,
  489. .CW Runesync
  490. was hexadecimal 21 and
  491. .CW Runeself
  492. was A0.)
  493. .CW UTFmax
  494. bytes are sufficient
  495. to hold the UTF encoding of any Unicode character.
  496. Characters of value less than
  497. .CW Runesync
  498. only appear in a UTF string as
  499. themselves, never as part of a sequence encoding another character.
  500. Characters of value less than
  501. .CW Runeself
  502. encode into single bytes
  503. of the same value.
  504. Finally, when the library detects errors in UTF input\(embyte sequences
  505. that are not valid UTF sequences\(emit converts the first byte of the
  506. error sequence to the character
  507. .CW Runeerror .
  508. There is little a rune-oriented program can do when given bad data
  509. except exit, which is unreasonable, or carry on.
  510. Originally the conversion routines, described below,
  511. returned errors when given invalid UTF,
  512. but we found ourselves repeatedly checking for errors and ignoring them.
  513. We therefore decided to convert a bad sequence to a valid rune
  514. and continue processing.
  515. (The ANSI C routines, on the other hand, return errors.)
  516. .PP
  517. This technique does have the unfortunate property that converting
  518. invalid UTF byte strings in and out of runes does not preserve the input,
  519. but this circumstance only occurs when non-textual input is
  520. given to a textual program.
  521. The Unicode Standard defines an error character, value FFFD, to stand for
  522. characters from other sets that it does not represent.
  523. The
  524. .CW Runeerror
  525. character is a different concept, related to the encoding rather than the character set, so we
  526. chose a different character for it.
  527. .PP
  528. The Plan 9 C library contains a number of routines for
  529. manipulating runes.
  530. The first set converts between runes and UTF strings:
  531. .P1
  532. extern int runetochar(char*, Rune*);
  533. extern int chartorune(Rune*, char*);
  534. extern int runelen(long);
  535. extern int fullrune(char*, int);
  536. .P2
  537. .CW Runetochar
  538. translates a single
  539. .CW Rune
  540. to a UTF sequence and returns the number of bytes produced.
  541. .CW Chartorune
  542. goes the other way, reporting how many bytes were consumed.
  543. .CW Runelen
  544. returns the number of bytes in the UTF encoding of a rune.
  545. .CW Fullrune
  546. examines a UTF string up to a specified number of bytes
  547. and reports whether the string begins with a complete UTF encoding.
  548. All these routines use the
  549. .CW Runeerror
  550. character to work around encoding problems.
  551. .PP
  552. There is also a set of routines for examining null-terminated UTF strings,
  553. based on the model of the ANSI standard
  554. .CW str
  555. routines, but with
  556. .CW utf
  557. substituted for
  558. .CW str
  559. and
  560. .CW rune
  561. for
  562. .CW chr :
  563. .P1
  564. extern int utflen(char*);
  565. extern char* utfrune(char*, long);
  566. extern char* utfrrune(char*, long);
  567. extern char* utfutf(char*, char*);
  568. .P2
  569. .CW Utflen
  570. returns the number of runes in a UTF string;
  571. .CW utfrune
  572. returns a pointer to the first occurrence of a rune in a UTF string;
  573. and
  574. .CW utfrrune
  575. a pointer to the last.
  576. .CW Utfutf
  577. searches for the first occurrence of a UTF string in another UTF string.
  578. Given the synchronizing property of UTF-8,
  579. .CW utfutf
  580. is the same as
  581. .CW strstr
  582. if the arguments point to valid UTF strings.
  583. .PP
  584. It is a mistake to use
  585. .CW strchr
  586. or
  587. .CW strrchr
  588. unless searching for a 7-bit ASCII character, that is, a character
  589. less than
  590. .CW Runeself .
  591. .PP
  592. We have no routines for manipulating null-terminated arrays of
  593. .CW Runes .
  594. Although they should probably exist for completeness, we have
  595. found no need for them, for the same reason that
  596. .CW %S
  597. and
  598. .CW L"..."
  599. are rarely used.
  600. .PP
  601. Most Plan 9 programs use a new buffered I/O library, BIO, in place of
  602. Standard I/O.
  603. BIO contains routines to read and write UTF streams, converting to and from
  604. runes.
  605. .CW Bgetrune
  606. returns, as a
  607. .CW Rune
  608. within a
  609. .CW long ,
  610. the next character in the UTF input stream;
  611. .CW Bputrune
  612. takes a rune and writes its UTF representation.
  613. .CW Bungetrune
  614. puts a rune back into the input stream for rereading.
  615. .PP
  616. Plan 9 programs use a simple set of macros to process command line arguments.
  617. Converting these macros to UTF automatically updated the
  618. argument processing of most programs.
  619. In general,
  620. argument flag names can no longer be held in bytes and
  621. arrays of 256 bytes cannot be used to hold a set of flags.
  622. .PP
  623. We have done nothing analogous to ANSI C's locales, partly because
  624. we do not feel qualified to define locales and partly because we remain
  625. unconvinced of that model for dealing with the problems.
  626. That is really more an issue of internationalization than conversion
  627. to a larger character set; on the other hand,
  628. because we have chosen a single character set that encompasses
  629. most languages, some of the need for
  630. locales is eliminated.
  631. (We have a utility,
  632. .CW tcs ,
  633. that translates between UTF and other character sets.)
  634. .PP
  635. There are several reasons why our library does not follow the ANSI design
  636. for wide and multi-byte characters.
  637. The ANSI model was designed by a committee, untried, almost
  638. as an afterthought, whereas
  639. we wanted to design as we built.
  640. (We made several major changes to the interface
  641. as we became familiar with the problems involved.)
  642. We disagree with ANSI C's handling of invalid multi-byte sequences.
  643. Also, the ANSI C library is incomplete:
  644. although it contains some crucial routines for handling
  645. wide and multi-byte characters, there are some serious omissions.
  646. For example, our software can exploit
  647. the fact that UTF preserves ASCII characters in the byte stream.
  648. We could remove that assumption by replacing all
  649. calls to
  650. .CW strchr
  651. with
  652. .CW utfrune
  653. and so on.
  654. (Because of the weaker properties of the original UTF,
  655. we have actually done so.)
  656. ANSI C cannot:
  657. the standard says nothing about the representation, so portable code should
  658. .I never
  659. call
  660. .CW strchr ,
  661. yet there is no ANSI equivalent to
  662. .CW utfrune .
  663. ANSI C simultaneously invalidates
  664. .CW strchr
  665. and offers no replacement.
  666. .PP
  667. Finally, ANSI did nothing to integrate wide characters
  668. into the I/O system: it gives no method for printing
  669. wide characters.
  670. We therefore needed to invent some things and decided to invent
  671. everything.
  672. In the end, some of our entry points do correspond closely to
  673. ANSI routines\(emfor example
  674. .CW chartorune
  675. and
  676. .CW runetochar
  677. are similar to
  678. .CW mbtowc
  679. and
  680. .CW wctomb \(embut
  681. Plan 9's library defines more functionality, enough
  682. to write real applications comfortably.
  683. .SH
  684. Converting the tools
  685. .PP
  686. The source for our tools and applications had already been converted to
  687. work with Latin-1, so it was `8-bit safe', but the conversion to the Unicode
  688. Standard and UTF is more involved.
  689. Some programs needed no change at all:
  690. .CW cat ,
  691. for instance,
  692. interprets its argument strings, delivered in UTF,
  693. as file names that it passes uninterpreted to the
  694. .CW open
  695. system call,
  696. and then just copies bytes from its input to its output;
  697. it never makes decisions based on the values of the bytes.
  698. (Plan 9
  699. .CW cat
  700. has no options such as
  701. .CW -v
  702. to complicate matters.)
  703. Most programs, however, needed modest change.
  704. .PP
  705. It is difficult to
  706. find automatically the places that need attention,
  707. but
  708. .CW grep
  709. helps.
  710. Software that uses the libraries conscientiously can be searched
  711. for calls to library routines that examine bytes as characters:
  712. .CW strchr ,
  713. .CW strrchr ,
  714. .CW strstr ,
  715. etc.
  716. Replacing these by calls to
  717. .CW utfrune ,
  718. .CW utfrrune ,
  719. and
  720. .CW utfutf
  721. is enough to fix many programs.
  722. Few tools actually need to operate on runes internally;
  723. more typically they need only to look for the final slash in a file
  724. name and similar trivial tasks.
  725. Of the 170 C source programs in the top levels of
  726. .CW /sys/src/cmd ,
  727. only 23 now contain the word
  728. .CW Rune .
  729. .PP
  730. The programs that
  731. .I do
  732. store runes internally
  733. are mostly those whose
  734. .I raison
  735. .I d'être
  736. is character manipulation:
  737. .CW sam
  738. (the text editor),
  739. .CW sed ,
  740. .CW sort ,
  741. .CW tr ,
  742. .CW troff ,
  743. .CW 8½
  744. (the window system and terminal emulator),
  745. and so on.
  746. To decide whether to compute using runes
  747. or UTF-encoded byte strings requires balancing the cost of converting
  748. the data when read and written
  749. against the cost of converting relevant text on demand.
  750. For programs such as editors that run a long time with a relatively
  751. constant dataset, runes are the better choice.
  752. There are space considerations too, but they are more complicated:
  753. plain ASCII text grows when converted to runes; UTF-encoded Japanese
  754. shrinks.
  755. .PP
  756. Again, it is hard to automate the conversion of a program from
  757. .CW chars
  758. to
  759. .CW Runes .
  760. It is not enough just to change the type of variables; the assumption
  761. that bytes and characters are equivalent can be insidious.
  762. For instance, to clear a character array by
  763. .P1
  764. memset(buf, 0, BUFSIZE)
  765. .P2
  766. becomes wrong if
  767. .CW buf
  768. is changed from an array of
  769. .CW chars
  770. to an array of
  771. .CW Runes .
  772. Any program that indexes tables based on character values needs
  773. rethinking.
  774. Consider
  775. .CW tr ,
  776. which originally used multiple 256-byte arrays for the mapping.
  777. The naïve conversion would yield multiple 65536-rune arrays.
  778. Instead Plan 9
  779. .CW tr
  780. saves space by building in effect
  781. a run-encoded version of the map.
  782. .PP
  783. .CW Sort
  784. has related problems.
  785. The cooperation of UTF and
  786. .CW strcmp
  787. means that a simple sort\(emone with no options\(emcan be done
  788. on the original UTF strings using
  789. .CW strcmp .
  790. With sorting options enabled, however,
  791. .CW sort
  792. may need to convert its input to runes: for example,
  793. option
  794. .CW -t\f1α\fP
  795. requires searching for alphas in the input text to
  796. crack the input into fields.
  797. The field specifier
  798. .CW +3.2
  799. refers to 2 runes beyond the third field.
  800. Some of the other options are hopelessly provincial:
  801. consider the case-folding and dictionary order options
  802. (Japanese doesn't even have an official dictionary order) or
  803. .CW -M
  804. which compares by case-insensitive English month name.
  805. Handling these options involves the
  806. larger issues of internationalization and is beyond the scope
  807. of this paper and our expertise.
  808. Plan 9
  809. .CW sort
  810. works sensibly with options that make sense relative to the input.
  811. The simple and most important options are, however, usually meaningful.
  812. In particular,
  813. .CW sort
  814. sorts UTF into the same order that
  815. .CW look
  816. expects.
  817. .PP
  818. Regular expression-matching algorithms need rethinking to
  819. be applied to UTF text.
  820. Deterministic automata are usually applied to bytes;
  821. converting them to operate on variable-sized byte sequences is awkward.
  822. On the other hand, converting the input stream to runes adds measurable
  823. expense
  824. and the state tables expand
  825. from size 256 to 65536; it can be expensive just to generate them.
  826. For simple string searching,
  827. the Boyer-Moore algorithm works with UTF provided the input is
  828. guaranteed to be only valid UTF strings; however, it does not work
  829. with the old UTF encoding.
  830. At a more mundane level, even character classes are harder:
  831. the usual bit-vector representation within a non-deterministic automaton
  832. is unwieldy with 65536 characters in the alphabet.
  833. .PP
  834. We compromised.
  835. An existing library for compiling and executing regular expressions
  836. was adapted to work on runes, with two entry points for searching
  837. in arrays of runes and arrays of chars (the pattern is always UTF text).
  838. Character classes are represented internally as runs of runes;
  839. the reserved value
  840. .CW FFFF
  841. marks the end of the class.
  842. Then
  843. .I all
  844. utilities that use regular expressions\(emeditors,
  845. .CW grep ,
  846. .CW awk ,
  847. etc.\(emexcept the shell, whose notation
  848. was grandfathered, were converted to use the library.
  849. For some programs, there was a concomitant loss of performance,
  850. but there was also a strong advantage.
  851. To our knowledge, Plan 9 is the only Unix-like system
  852. that has a single definition and implementation of
  853. regular expressions; patterns are written and interpreted
  854. identically by all the programs in the system.
  855. .PP
  856. A handful of programs have the notion of character built into them
  857. so strongly as to confuse the issue of what they should do with UTF input.
  858. Such programs were treated as individual special cases.
  859. For example,
  860. .CW wc
  861. is, by default, unchanged in behavior and output; a new option,
  862. .CW -r ,
  863. counts the number of correctly encoded runes\(emvalid UTF sequences\(emin
  864. its input;
  865. .CW -b
  866. the number of invalid sequences.
  867. .PP
  868. It took us several months to convert all the software in the system
  869. to the Unicode Standard and the old UTF.
  870. When we decided to convert from that to the new UTF,
  871. only three things needed to be done.
  872. First, we rewrote the library routines to encode and decode the
  873. new UTF. This took an evening.
  874. Next, we converted all the files containing UTF
  875. to the new encoding.
  876. We wrote a trivial program to look for non-ASCII bytes in
  877. text files and used a Plan 9 program called
  878. .CW tcs
  879. (translate character set) to change encodings.
  880. Finally, we recompiled all the system software;
  881. the library interface was unchanged, so recompilation was sufficient
  882. to effect the transformation.
  883. The second two steps were done concurrently and took an afternoon.
  884. We concluded that the actual encoding is relatively unimportant to the
  885. software; the adoption of large characters and a byte-stream encoding
  886. .I per
  887. .I se
  888. are much deeper issues.
  889. .SH
  890. Graphics and fonts
  891. .PP
  892. Plan 9 provides only minimal support for plain text terminals.
  893. It is instead designed to be used with all character input and
  894. output mediated by a window system such as
  895. .CW 8½ .
  896. The window system and related software are responsible for the
  897. display of UTF text as Unicode character images.
  898. For plain text, the window system must provide a user-settable
  899. .I font
  900. that provides a (possibly empty) picture for each Unicode character.
  901. Fancier applications that use bold and Italic characters
  902. need multiple fonts storing multiple pictures for each
  903. Unicode value.
  904. All the issues are apparent, though,
  905. in just the problem of
  906. displaying a single image for each character, that is, the
  907. Unicode equivalent of a plain text terminal.
  908. With 128 or even 256 characters, a font can be just
  909. an array of bitmaps. With 65536 characters,
  910. a more sophisticated design is necessary. To store the ideographs
  911. for just Japanese as 16×16×1 bit images,
  912. the smallest they can reasonably be, takes over a quarter of a
  913. megabyte. Make the images a little larger, store more bits per
  914. pixel, and hold a copy in every running application, and the
  915. memory cost becomes unreasonable.
  916. .PP
  917. The structure of the bitmap graphics services is described at length elsewhere
  918. [Pike91].
  919. In summary, the memory holding the bitmaps is stored in the same machine that has
  920. the display, mouse, and keyboard: the terminal in Plan 9 terminology,
  921. the workstation in others'.
  922. Access to that memory and associated services is provided
  923. by device files served by system
  924. software on the terminal. One of those files,
  925. .CW /dev/bitblt ,
  926. interprets messages written upon it as requests for actions
  927. corresponding to entry points in the graphics library:
  928. allocate a bitmap, execute a raster operation, draw a text string, etc.
  929. The window system
  930. acts as a multiplexer that mediates access to the services
  931. and resources of the terminal by simulating in each client window
  932. a set of files mirroring those provided by the system.
  933. That is, each window has a distinct
  934. .CW /dev/mouse ,
  935. .CW /dev/bitblt ,
  936. and so on through which applications drive graphical
  937. input and output.
  938. .PP
  939. One of the resources managed by
  940. .CW 8½
  941. and the terminal is the set of active
  942. .I subfonts.
  943. Each subfont holds the
  944. bitmaps and associated data structures for a sequential set of Unicode
  945. characters.
  946. Subfonts are stored in files and loaded into the terminal by
  947. .CW 8½
  948. or an application.
  949. For example, one subfont
  950. might hold the images of the first 256 characters of the Unicode space,
  951. corresponding to the Latin-1 character set;
  952. another might hold the standard phonetic character set, Unicode characters
  953. with value 0250 to 02E9.
  954. These files are collected in directories corresponding to typefaces:
  955. .CW /lib/font/bit/pelm
  956. contains the Pellucida Monospace character set, with subfonts holding
  957. the Latin-1, Greek, Cyrillic and other components of the typeface.
  958. A suffix on subfont files encodes (in a subfont-specific
  959. way) the size of the images:
  960. .CW /lib/font/bit/pelm/latin1.9
  961. contains the Latin-1 Pellucida Monospace characters with lower
  962. case letters 9 pixels high;
  963. .CW /lib/font/bit/jis/jis5400.16
  964. contains 16-pixel high
  965. ideographs starting at Unicode value 5400.
  966. .PP
  967. The subfonts do not identify which portion of the Unicode space
  968. they cover. Instead, a
  969. font file, in plain text,
  970. describes how to assemble subfonts into a complete
  971. character set.
  972. The font file is presented as an argument to the window system
  973. to determine how plain text is displayed in text windows and
  974. applications.
  975. Here is the beginning of the font file
  976. .CW /lib/font/bit/pelm/jis.9.font ,
  977. which describes the layout of a font covering that portion of
  978. the Unicode Standard for which we have characters of typical
  979. display size, using Japanese characters
  980. to cover the Han space:
  981. .P1
  982. 18 14
  983. 0x0000 0x00FF latin1.9
  984. 0x0100 0x017E latineur.9
  985. 0x0250 0x02E9 ipa.9
  986. 0x0386 0x03F5 greek.9
  987. 0x0400 0x0475 cyrillic.9
  988. 0x2000 0x2044 ../misc/genpunc.9
  989. 0x2070 0x208E supsub.9
  990. 0x20A0 0x20AA currency.9
  991. 0x2100 0x2138 ../misc/letterlike.9
  992. 0x2190 0x21EA ../misc/arrows
  993. 0x2200 0x227F ../misc/math1
  994. 0x2280 0x22F1 ../misc/math2
  995. 0x2300 0x232C ../misc/tech
  996. 0x2500 0x257F ../misc/chart
  997. 0x2600 0x266F ../misc/ding
  998. .P2
  999. .P1
  1000. 0x3000 0x303f ../jis/jis3000.16
  1001. 0x30a1 0x30fe ../jis/katakana.16
  1002. 0x3041 0x309e ../jis/hiragana.16
  1003. 0x4e00 0x4fff ../jis/jis4e00.16
  1004. 0x5000 0x51ff ../jis/jis5000.16
  1005. \&...
  1006. .P2
  1007. The first two numbers set the interline spacing of the font (18
  1008. pixels) and the distance from the baseline to the top of the
  1009. line (14 pixels).
  1010. When characters are displayed, they are placed so as best
  1011. to fit within those constraints; characters
  1012. too large to fit will be truncated.
  1013. The rest of the file associates subfont files
  1014. with portions of Unicode space.
  1015. The first four such files are in the Pellucida Monospace typeface
  1016. and directory; others reside in other directories. The file names
  1017. are relative to the font file's own location.
  1018. .PP
  1019. There are several advantages to this two-level structure.
  1020. First, it simultaneously breaks the huge Unicode space into manageable
  1021. components and provides a unifying architecture for
  1022. assembling fonts from disjoint pieces.
  1023. Second, the structure promotes sharing.
  1024. For example, we have only one set of Japanese
  1025. characters but dozens of typefaces for the Latin-1 characters,
  1026. and this structure permits us to store only one copy of the
  1027. Japanese set but use it with any Roman typeface.
  1028. Also, customization is easy.
  1029. English-speaking users who don't need Japanese characters
  1030. but may want to read an on-line Oxford English Dictionary can
  1031. assemble a custom font with the
  1032. Latin-1 (or even just ASCII) characters and the International
  1033. Phonetic Alphabet (IPA).
  1034. Moreover, to do so requires just editing a plain text file,
  1035. not using a special font editing tool.
  1036. Finally, the structure guides the design of
  1037. caching protocols to improve performance and memory usage.
  1038. .PP
  1039. To load a complete Unicode character set into each application
  1040. would consume too
  1041. much memory and, particularly on slow terminal lines, would take
  1042. unreasonably long.
  1043. Instead, Plan 9 assembles a multi-level cache structure for
  1044. each font.
  1045. An application opens a font file, reads and parses it,
  1046. and allocates a data structure.
  1047. A message written to
  1048. .CW /dev/bitblt
  1049. allocates an associated structure held in the terminal, in particular,
  1050. a bitmap to act as a cache
  1051. for recently used character images.
  1052. Other messages copy these images to bitmaps such as the screen
  1053. by loading characters from subfonts into the cache on demand and
  1054. from there to the destination bitmap.
  1055. The protocol to draw characters is in terms of cache indices,
  1056. not Unicode character number or UTF sequences.
  1057. These details are hidden from the application, which instead
  1058. sees only a subroutine to draw a string in a bitmap from a
  1059. given font, functions to discover character size information,
  1060. and routines to allocate and to free fonts.
  1061. .PP
  1062. As needed, whole
  1063. subfonts are opened by the graphics library, read, and then downloaded
  1064. to the terminal.
  1065. They are held open by the library in an LRU-replacement list.
  1066. Even when the program closes a subfont, it is retained
  1067. in the terminal for later use.
  1068. When the application opens the subfont, it asks the terminal
  1069. if it already has a copy to avoid reading it from the file
  1070. server if possible.
  1071. This level of cache has the property that the bitmaps for, say,
  1072. all the Japanese characters are stored only once, in the terminal;
  1073. the applications read only size and width information from the terminal
  1074. and share the images.
  1075. .PP
  1076. The sizes of the character and subfont caches held by the
  1077. application are adaptive.
  1078. A simple algorithm monitors the cache miss rate to enlarge and
  1079. shrink the caches as required.
  1080. The size of the character cache is limited to 2048 images maximum,
  1081. which in practice seems enough even for Japanese text.
  1082. For plain ASCII-like text it naturally stays around 128 images.
  1083. .PP
  1084. This mechanism sounds complicated but is implemented by only about
  1085. 500 lines in the library and considerably less in each of the
  1086. terminal's graphics driver and
  1087. .CW 8½ .
  1088. It has the advantage that only characters that are
  1089. being used are loaded into memory.
  1090. It is also efficient: if the characters being drawn
  1091. are in the cache the extra overhead is negligible.
  1092. It works particularly well for alphabetic character sets,
  1093. but also adapts on demand for ideographic sets.
  1094. When a user first looks at Japanese text, it takes a few
  1095. seconds to read all the font data, but thereafter the
  1096. text is drawn almost as fast as regular text (the images
  1097. are larger, so draw a little slower).
  1098. Also, because the bitmaps are remembered by the terminal,
  1099. if a second application then looks at Japanese text
  1100. it starts faster than the first.
  1101. .PP
  1102. We considered
  1103. building a `font server'
  1104. to cache character images and associated data
  1105. for the applications, the window system, and the terminal.
  1106. We rejected this design because, although isolating
  1107. many of the problems of font management into a separate program,
  1108. it didn't simplify the applications.
  1109. Moreover, in a distributed system such as Plan 9 it is easy
  1110. to have too many special purpose servers.
  1111. Making the management of the fonts the concern of only
  1112. the essential components simplifies the system and makes
  1113. bootstrapping less intricate.
  1114. .SH
  1115. Input
  1116. .PP
  1117. A completely different problem is how to type Unicode characters
  1118. as input to the system.
  1119. We selected an unused key on our ASCII keyboards
  1120. to serve as a prefix for multi-keystroke
  1121. sequences that generate Unicode characters.
  1122. For example, the character
  1123. .CW ü
  1124. is generated by the prefix key
  1125. (typically
  1126. .CW ALT
  1127. or
  1128. .CW Compose )
  1129. followed by a double quote and a lower-case
  1130. .CW u .
  1131. When that character is read by the application, from the file
  1132. .CW /dev/cons ,
  1133. it is of course presented as its UTF encoding.
  1134. Such sequences generate characters from an arbitrary set that
  1135. includes all of Latin-1 plus a selection of mathematical
  1136. and technical characters.
  1137. An arbitrary Unicode character may be generated by typing the prefix,
  1138. an upper case X, and four hexadecimal digits that identify
  1139. the Unicode value.
  1140. .PP
  1141. These simple mechanisms are adequate for most of our day-to-day needs:
  1142. it's easy to remember to type `ALT 1 2' for ½\^ or `ALT accent letter'
  1143. for accented Latin letters.
  1144. For the occasional unusual character, the cut and paste features of
  1145. .CW 8½
  1146. serve well. A program called (perhaps misleadingly)
  1147. .CW unicode
  1148. takes as argument a hexadecimal value, and prints the UTF representation of that character,
  1149. which may then be picked up with the mouse and used as input.
  1150. .PP
  1151. These methods
  1152. are clearly unsatisfactory when working in a non-English language.
  1153. In the native country of such a language
  1154. the appropriate keyboard is likely to be at hand.
  1155. But it's also reasonable\(emespecially now that the system handles Unicode characters\(emto
  1156. work in a language foreign to the keyboard.
  1157. .PP
  1158. For alphabetic languages such as Greek or Russian, it is
  1159. straightforward to construct a program that does phonetic substitution,
  1160. so that, for example, typing a Latin `a' yields the Greek `α'.
  1161. Within Plan 9, such a program can be inserted transparently
  1162. between the real keyboard and a program such as the window system,
  1163. providing a manageable input device for such languages.
  1164. .PP
  1165. For ideographic languages such as Chinese or Japanese the problem is harder.
  1166. Native users of such languages have adopted methods for dealing with
  1167. Latin keyboards that involve a hybrid technique based on phonetics
  1168. to generate a list of possible symbols followed by menu selection to
  1169. choose the desired one.
  1170. Such methods can be
  1171. effective, but their design must be rooted in information about
  1172. the language unknown to non-native speakers.
  1173. .CW Cxterm , (
  1174. a Chinese terminal emulator built by and for
  1175. Chinese programmers,
  1176. employs such a technique
  1177. [Pong and Zhang].)
  1178. Although the technical problem of implementing such a device
  1179. is easy in Plan 9\(emit is just an elaboration of the technique for
  1180. alphabetic languages\(emour lack of familiarity with such languages
  1181. has restrained our enthusiasm for building one.
  1182. .PP
  1183. The input problem is technically the least interesting but perhaps
  1184. emotionally the most important of the problems of converting a system
  1185. to an international character set.
  1186. Beyond that remain the deeper problems of internationalization
  1187. such as multi-lingual error messages and command names,
  1188. problems we are not qualified to solve.
  1189. With the ability to treat text of most languages on an equal
  1190. footing, though, we can begin down that path.
  1191. Perhaps people in non-English speaking countries will
  1192. consider adopting Plan 9, solving the input problem locally\(emperhaps
  1193. just by plugging in their local terminals\(emand begin to use
  1194. a system with at least the capacity to be international.
  1195. .SH
  1196. Acknowledgements
  1197. .PP
  1198. Dennis Ritchie provided consultation and encouragement.
  1199. Bob Flandrena converted most of the standard tools to UTF.
  1200. Brian Kernighan suffered cheerfully with several
  1201. inadequate implementations and converted
  1202. .CW troff
  1203. to UTF.
  1204. Rich Drechsler converted his Postscript driver to UTF.
  1205. John Hobby built the Postscript ☺.
  1206. We thank them all.
  1207. .SH
  1208. References
  1209. .LP
  1210. [ANSIC] \f2American National Standard for Information Systems \-
  1211. Programming Language C\f1, American National Standards Institute, Inc.,
  1212. New York, 1990.
  1213. .LP
  1214. [ISO10646]
  1215. ISO/IEC DIS 10646-1:1993
  1216. \f2Information technology \-
  1217. Universal Multiple-Octet Coded Character Set (UCS) \(em
  1218. Part 1: Architecture and Basic Multilingual Plane\fP.
  1219. .LP
  1220. [Pike90] R. Pike, D. Presotto, K. Thompson, H. Trickey,
  1221. ``Plan 9 from Bell Labs'',
  1222. UKUUG Proc. of the Summer 1990 Conf.,
  1223. London, England,
  1224. 1990.
  1225. .LP
  1226. [Pike91] R. Pike, ``8½, The Plan 9 Window System'', USENIX Summer
  1227. Conf. Proc., Nashville, 1991, reprinted in this volume.
  1228. .LP
  1229. [Pike92] R. Pike, ``How to Use the Plan 9 C Compiler'', this volume.
  1230. .LP
  1231. [Pong and Zhang] Man-Chi Pong and Yongguang Zhang, ``cxterm:
  1232. A Chinese Terminal Emulator for the X Window System'',
  1233. .I
  1234. Software\(emPractice and Experience,
  1235. .R
  1236. Vol 22(1), 809-926, October 1992.
  1237. .LP
  1238. [Unicode]
  1239. \f2The Unicode Standard,
  1240. Worldwide Character Encoding,
  1241. Version 1.0, Volume 1\f1,
  1242. The Unicode Consortium,
  1243. Addison Wesley,
  1244. New York,
  1245. 1991.