sam.ms 92 KB

1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465666768697071727374757677787980818283848586878889909192939495969798991001011021031041051061071081091101111121131141151161171181191201211221231241251261271281291301311321331341351361371381391401411421431441451461471481491501511521531541551561571581591601611621631641651661671681691701711721731741751761771781791801811821831841851861871881891901911921931941951961971981992002012022032042052062072082092102112122132142152162172182192202212222232242252262272282292302312322332342352362372382392402412422432442452462472482492502512522532542552562572582592602612622632642652662672682692702712722732742752762772782792802812822832842852862872882892902912922932942952962972982993003013023033043053063073083093103113123133143153163173183193203213223233243253263273283293303313323333343353363373383393403413423433443453463473483493503513523533543553563573583593603613623633643653663673683693703713723733743753763773783793803813823833843853863873883893903913923933943953963973983994004014024034044054064074084094104114124134144154164174184194204214224234244254264274284294304314324334344354364374384394404414424434444454464474484494504514524534544554564574584594604614624634644654664674684694704714724734744754764774784794804814824834844854864874884894904914924934944954964974984995005015025035045055065075085095105115125135145155165175185195205215225235245255265275285295305315325335345355365375385395405415425435445455465475485495505515525535545555565575585595605615625635645655665675685695705715725735745755765775785795805815825835845855865875885895905915925935945955965975985996006016026036046056066076086096106116126136146156166176186196206216226236246256266276286296306316326336346356366376386396406416426436446456466476486496506516526536546556566576586596606616626636646656666676686696706716726736746756766776786796806816826836846856866876886896906916926936946956966976986997007017027037047057067077087097107117127137147157167177187197207217227237247257267277287297307317327337347357367377387397407417427437447457467477487497507517527537547557567577587597607617627637647657667677687697707717727737747757767777787797807817827837847857867877887897907917927937947957967977987998008018028038048058068078088098108118128138148158168178188198208218228238248258268278288298308318328338348358368378388398408418428438448458468478488498508518528538548558568578588598608618628638648658668678688698708718728738748758768778788798808818828838848858868878888898908918928938948958968978988999009019029039049059069079089099109119129139149159169179189199209219229239249259269279289299309319329339349359369379389399409419429439449459469479489499509519529539549559569579589599609619629639649659669679689699709719729739749759769779789799809819829839849859869879889899909919929939949959969979989991000100110021003100410051006100710081009101010111012101310141015101610171018101910201021102210231024102510261027102810291030103110321033103410351036103710381039104010411042104310441045104610471048104910501051105210531054105510561057105810591060106110621063106410651066106710681069107010711072107310741075107610771078107910801081108210831084108510861087108810891090109110921093109410951096109710981099110011011102110311041105110611071108110911101111111211131114111511161117111811191120112111221123112411251126112711281129113011311132113311341135113611371138113911401141114211431144114511461147114811491150115111521153115411551156115711581159116011611162116311641165116611671168116911701171117211731174117511761177117811791180118111821183118411851186118711881189119011911192119311941195119611971198119912001201120212031204120512061207120812091210121112121213121412151216121712181219122012211222122312241225122612271228122912301231123212331234123512361237123812391240124112421243124412451246124712481249125012511252125312541255125612571258125912601261126212631264126512661267126812691270127112721273127412751276127712781279128012811282128312841285128612871288128912901291129212931294129512961297129812991300130113021303130413051306130713081309131013111312131313141315131613171318131913201321132213231324132513261327132813291330133113321333133413351336133713381339134013411342134313441345134613471348134913501351135213531354135513561357135813591360136113621363136413651366136713681369137013711372137313741375137613771378137913801381138213831384138513861387138813891390139113921393139413951396139713981399140014011402140314041405140614071408140914101411141214131414141514161417141814191420142114221423142414251426142714281429143014311432143314341435143614371438143914401441144214431444144514461447144814491450145114521453145414551456145714581459146014611462146314641465146614671468146914701471147214731474147514761477147814791480148114821483148414851486148714881489149014911492149314941495149614971498149915001501150215031504150515061507150815091510151115121513151415151516151715181519152015211522152315241525152615271528152915301531153215331534153515361537153815391540154115421543154415451546154715481549155015511552155315541555155615571558155915601561156215631564156515661567156815691570157115721573157415751576157715781579158015811582158315841585158615871588158915901591159215931594159515961597159815991600160116021603160416051606160716081609161016111612161316141615161616171618161916201621162216231624162516261627162816291630163116321633163416351636163716381639164016411642164316441645164616471648164916501651165216531654165516561657165816591660166116621663166416651666166716681669167016711672167316741675167616771678167916801681168216831684168516861687168816891690169116921693169416951696169716981699170017011702170317041705170617071708170917101711171217131714171517161717171817191720172117221723172417251726172717281729173017311732173317341735173617371738173917401741174217431744174517461747174817491750175117521753175417551756175717581759176017611762176317641765176617671768176917701771177217731774177517761777177817791780178117821783178417851786178717881789179017911792179317941795179617971798179918001801180218031804180518061807180818091810181118121813181418151816181718181819182018211822182318241825182618271828182918301831183218331834183518361837183818391840184118421843184418451846184718481849185018511852185318541855185618571858185918601861186218631864186518661867186818691870187118721873187418751876187718781879188018811882188318841885188618871888188918901891189218931894189518961897189818991900190119021903190419051906190719081909191019111912191319141915191619171918191919201921192219231924192519261927192819291930193119321933193419351936193719381939194019411942194319441945194619471948194919501951195219531954195519561957195819591960196119621963196419651966196719681969197019711972197319741975197619771978197919801981198219831984198519861987198819891990199119921993199419951996199719981999200020012002200320042005200620072008200920102011201220132014201520162017201820192020202120222023202420252026202720282029203020312032203320342035203620372038203920402041204220432044204520462047204820492050205120522053205420552056205720582059206020612062206320642065206620672068206920702071207220732074207520762077207820792080208120822083208420852086208720882089209020912092209320942095209620972098209921002101210221032104210521062107210821092110211121122113211421152116211721182119212021212122212321242125212621272128212921302131213221332134213521362137213821392140214121422143214421452146214721482149215021512152215321542155215621572158215921602161216221632164216521662167216821692170217121722173217421752176217721782179218021812182218321842185218621872188218921902191219221932194219521962197219821992200220122022203220422052206220722082209221022112212221322142215221622172218221922202221222222232224222522262227222822292230223122322233223422352236223722382239224022412242224322442245224622472248224922502251225222532254225522562257225822592260226122622263226422652266226722682269227022712272227322742275227622772278227922802281228222832284228522862287228822892290229122922293229422952296229722982299230023012302230323042305230623072308230923102311231223132314231523162317231823192320232123222323232423252326232723282329233023312332233323342335233623372338233923402341234223432344234523462347234823492350235123522353235423552356235723582359236023612362236323642365236623672368236923702371237223732374237523762377237823792380238123822383238423852386238723882389239023912392239323942395239623972398239924002401240224032404240524062407240824092410241124122413241424152416241724182419242024212422242324242425242624272428242924302431243224332434243524362437243824392440244124422443244424452446244724482449245024512452245324542455245624572458245924602461246224632464246524662467246824692470247124722473247424752476247724782479248024812482248324842485248624872488248924902491249224932494249524962497249824992500250125022503250425052506250725082509251025112512251325142515251625172518251925202521252225232524252525262527252825292530253125322533253425352536253725382539254025412542254325442545254625472548254925502551255225532554255525562557255825592560256125622563256425652566256725682569257025712572257325742575257625772578257925802581258225832584258525862587258825892590259125922593259425952596259725982599260026012602260326042605260626072608260926102611261226132614261526162617261826192620262126222623262426252626262726282629263026312632263326342635263626372638263926402641264226432644264526462647264826492650265126522653265426552656265726582659266026612662266326642665266626672668266926702671267226732674267526762677267826792680268126822683268426852686268726882689269026912692269326942695269626972698269927002701270227032704270527062707270827092710271127122713271427152716271727182719272027212722272327242725272627272728272927302731273227332734273527362737273827392740274127422743274427452746274727482749275027512752275327542755275627572758275927602761276227632764276527662767276827692770277127722773277427752776277727782779278027812782278327842785278627872788278927902791279227932794279527962797279827992800280128022803280428052806280728082809281028112812281328142815281628172818281928202821282228232824282528262827282828292830283128322833283428352836283728382839284028412842284328442845284628472848284928502851285228532854285528562857285828592860286128622863286428652866286728682869287028712872287328742875287628772878287928802881288228832884288528862887288828892890289128922893289428952896289728982899290029012902290329042905290629072908290929102911291229132914291529162917291829192920292129222923292429252926292729282929293029312932293329342935293629372938293929402941294229432944294529462947294829492950295129522953295429552956295729582959296029612962296329642965296629672968296929702971297229732974297529762977297829792980298129822983298429852986298729882989299029912992299329942995299629972998299930003001300230033004300530063007300830093010301130123013301430153016301730183019302030213022302330243025302630273028302930303031303230333034303530363037303830393040304130423043304430453046304730483049305030513052305330543055305630573058305930603061306230633064306530663067306830693070307130723073307430753076307730783079308030813082308330843085308630873088308930903091309230933094309530963097309830993100310131023103310431053106310731083109311031113112311331143115311631173118311931203121312231233124312531263127312831293130313131323133313431353136313731383139314031413142314331443145314631473148314931503151315231533154315531563157315831593160316131623163316431653166316731683169317031713172317331743175317631773178317931803181318231833184318531863187318831893190319131923193319431953196319731983199320032013202320332043205320632073208320932103211321232133214321532163217321832193220322132223223322432253226322732283229323032313232323332343235323632373238323932403241
  1. .HTML "The Text Editor sam
  2. .Vx 17 11 November 87 1 32 "ROB PIKE" "THE TEXT EDITOR SAM"
  3. .ds DY "31 May 1987
  4. .ds DR "Revised 1 July 1987
  5. .de CW \" puts first arg in CW font, same as UL; maintains font
  6. \%\&\\$3\f(CW\\$1\fP\&\\$2
  7. ..
  8. .de Cs
  9. .br
  10. .fi
  11. .ft 2
  12. .ps -2
  13. .vs -2
  14. ..
  15. .de Ce
  16. .br
  17. .nf
  18. .ft 1
  19. .ps
  20. .vs
  21. .sp
  22. ..
  23. .de XP
  24. .ie h .html - <center><img src="sam.\\$1.png" /></center>
  25. .el .BP \\$1.ps \\$2
  26. ..
  27. .TL
  28. The Text Editor \&\f(CWsam\fP
  29. .AU
  30. Rob Pike
  31. rob@plan9.bell-labs.com
  32. .AB
  33. .LP
  34. .CW Sam
  35. is an interactive multi-file text editor intended for
  36. bitmap displays.
  37. A textual command language
  38. supplements the mouse-driven, cut-and-paste interface
  39. to make complex or
  40. repetitive editing tasks easy to specify.
  41. The language is characterized by the composition of regular expressions
  42. to describe the structure of the text being modified.
  43. The treatment of files as a database, with changes logged
  44. as atomic transactions, guides the implementation and
  45. makes a general `undo' mechanism straightforward.
  46. .PP
  47. .CW Sam
  48. is implemented as two processes connected by a low-bandwidth stream,
  49. one process handling the display and the other the editing
  50. algorithms. Therefore it can run with the display process
  51. in a bitmap terminal and the editor on a local host,
  52. with both processes on a bitmap-equipped host, or with
  53. the display process in the terminal and the editor in a
  54. remote host.
  55. By suppressing the display process,
  56. it can even run without a bitmap terminal.
  57. .PP
  58. This paper is reprinted from Software\(emPractice and Experience,
  59. Vol 17, number 11, pp. 813-845, November 1987.
  60. The paper has not been updated for the Plan 9 manuals. Although
  61. .CW Sam
  62. has not changed much since the paper was written, the system around it certainly has.
  63. Nonetheless, the description here still stands as the best introduction to the editor.
  64. .AE
  65. .SH
  66. Introduction
  67. .LP
  68. .CW Sam
  69. is an interactive text editor that combines cut-and-paste interactive editing with
  70. an unusual command language based on the composition of regular expressions.
  71. It is written as two programs: one, the `host part,' runs on a UNIX system
  72. and implements the command language and provides file access; the other, the
  73. `terminal part,' runs asynchronously
  74. on a machine with a mouse and bitmap display
  75. and supports the display and interactive editing.
  76. The host part may be even run in isolation on an ordinary terminal
  77. to edit text using the command
  78. language, much like a traditional line editor,
  79. without assistance from a mouse or display.
  80. Most often,
  81. the terminal part runs on a Blit\u\s-4\&1\s+4\d terminal
  82. (actually on a Teletype DMD 5620, the production version of the Blit), whose
  83. host connection is an ordinary 9600 bps RS232 link;
  84. on the SUN computer the host and display processes run on a single machine,
  85. connected by a pipe.
  86. .PP
  87. .CW Sam
  88. edits uninterpreted
  89. ASCII text.
  90. It has no facilities for multiple fonts, graphics or tables,
  91. unlike MacWrite,\u\s-4\&2\s+4\d Bravo,\u\s-4\&3\s+4\d Tioga\u\s-4\&4\s+4\d
  92. or Lara.\u\s-4\&5\s+4\d
  93. Also unlike them, it has a rich command language.
  94. (Throughout this paper, the phrase
  95. .I
  96. command language
  97. .R
  98. refers to
  99. textual commands; commands activated from the mouse form the
  100. .I mouse
  101. .I language. )
  102. .CW Sam
  103. developed as an editor for use by programmers, and tries to join
  104. the styles of the UNIX text editor
  105. .CW ed \u\s-4\&6,7\s+4\d
  106. with that of interactive cut-and-paste editors by
  107. providing a comfortable mouse-driven interface
  108. to a program with a solid command language driven by regular expressions.
  109. The command language developed more than the mouse language, and
  110. acquired a notation for describing the structure of files
  111. more richly than as a sequence of lines,
  112. using a dataflow-like syntax for specifying changes.
  113. .PP
  114. The interactive style was influenced by
  115. .CW jim ,\u\s-4\&1\s+4\d
  116. an early cut-and-paste editor for the Blit, and by
  117. .CW mux ,\u\s-4\&8\s+4\d
  118. the Blit window system.
  119. .CW Mux
  120. merges the original Blit window system,
  121. .CW mpx ,\u\s-4\&1\s+4\d
  122. with cut-and-paste editing, forming something like a
  123. multiplexed version of
  124. .CW jim
  125. that edits the output of (and input to) command sessions rather than files.
  126. .PP
  127. The first part of this paper describes the command language, then the mouse
  128. language, and explains how they interact.
  129. That is followed by a description of the implementation,
  130. first of the host part, then of the terminal part.
  131. A principle that influenced the design of
  132. .CW sam
  133. is that it should have no explicit limits, such as upper limits on
  134. file size or line length.
  135. A secondary consideration is that it be efficient.
  136. To honor these two goals together requires a method for efficiently
  137. manipulating
  138. huge strings (files) without breaking them into lines,
  139. perhaps while making thousands of changes
  140. under control of the command language.
  141. .CW Sam 's
  142. method is to
  143. treat the file as a transaction database, implementing changes as atomic
  144. updates. These updates may be unwound easily to `undo' changes.
  145. Efficiency is achieved through a collection of caches that minimizes
  146. disc traffic and data motion, both within the two parts of the program
  147. and between them.
  148. .PP
  149. The terminal part of
  150. .CW sam
  151. is fairly straightforward.
  152. More interesting is how the two halves of the editor stay
  153. synchronized when either half may initiate a change.
  154. This is achieved through a data structure that organizes the
  155. communications and is maintained in parallel by both halves.
  156. .PP
  157. The last part of the paper chronicles the writing of
  158. .CW sam
  159. and discusses the lessons that were learned through its development and use.
  160. .PP
  161. The paper is long, but is composed largely of two papers of reasonable length:
  162. a description of the user interface of
  163. .CW sam
  164. and a discussion of its implementation.
  165. They are combined because the implementation is strongly influenced by
  166. the user interface, and vice versa.
  167. .SH
  168. The Interface
  169. .LP
  170. .CW Sam
  171. is a text editor for multiple files.
  172. File names may be provided when it is invoked:
  173. .P1
  174. sam file1 file2 ...
  175. .P2
  176. and there are commands
  177. to add new files and discard unneeded ones.
  178. Files are not read until necessary
  179. to complete some command.
  180. Editing operations apply to an internal copy
  181. made when the file is read; the UNIX file associated with the copy
  182. is changed only by an explicit command.
  183. To simplify the discussion, the internal copy is here called a
  184. .I file ,
  185. while the disc-resident original is called a
  186. .I
  187. disc file.
  188. .R
  189. .PP
  190. .CW Sam
  191. is usually connected to a bitmap display that presents a cut-and-paste
  192. editor driven by the mouse.
  193. In this mode, the command language is still available:
  194. text typed in a special window, called the
  195. .CW sam
  196. .I window,
  197. is interpreted
  198. as commands to be executed in the current file.
  199. Cut-and-paste editing may be used in any window \(em even in the
  200. .CW sam
  201. window to construct commands.
  202. The other mode of operation, invoked by starting
  203. .CW sam
  204. with the option
  205. .CW -d
  206. (for `no download'),
  207. does not use the mouse or bitmap display, but still permits
  208. editing using the textual command language, even on an ordinary terminal,
  209. interactively or from a script.
  210. .PP
  211. The following sections describe first the command language (under
  212. .CW sam\ -d
  213. and in the
  214. .CW sam
  215. window), and then the mouse interface.
  216. These two languages are nearly independent, but connect through the
  217. .I current
  218. .I text,
  219. described below.
  220. .SH 2
  221. The Command Language
  222. .LP
  223. A file consists of its contents, which are an array of characters
  224. (that is, a string); the
  225. .I name
  226. of the associated disc file; the
  227. .I
  228. modified bit
  229. .R
  230. that states whether the contents match those of
  231. the disc file;
  232. and a substring of the contents, called the
  233. .I
  234. current text
  235. .R
  236. or
  237. .I dot
  238. (see Figures 1 and 2).
  239. If the current text is a null string, dot falls between characters.
  240. The
  241. .I value
  242. of dot is the location of the current text; the
  243. .I contents
  244. of dot are the characters it contains.
  245. .CW Sam
  246. imparts to the text no two-dimensional interpretation such as columns
  247. or fields; text is always one-dimensional.
  248. Even the idea of a `line' of text as understood by most UNIX programs
  249. \(em a sequence of characters terminated by a newline character \(em
  250. is only weakly supported.
  251. .PP
  252. The
  253. .I
  254. current file
  255. .R
  256. is the file to which editing commands refer.
  257. The current text is therefore dot in the current file.
  258. If a command doesn't explicitly name a particular file or piece of text,
  259. the command is assumed to apply to the current text.
  260. For the moment, ignore the presence of multiple files and consider
  261. editing a single file.
  262. .KF L
  263. .XP fig1 3.5i
  264. .Cs
  265. Figure 1. A typical
  266. .CW sam
  267. screen, with the editing menu presented.
  268. The
  269. .CW sam
  270. (command language) window is in the middle, with file windows above and below.
  271. (The user interface makes it easy to create these abutting windows.)
  272. The partially obscured window is a third file window.
  273. The uppermost window is that to which typing and mouse operations apply,
  274. as indicated by its heavy border.
  275. Each window has its current text highlighted in reverse video.
  276. The
  277. .CW sam
  278. window's current text is the null string on the last visible line,
  279. indicated by a vertical bar.
  280. See also Figure 2.
  281. .Ce
  282. .KE
  283. .PP
  284. Commands have one-letter names.
  285. Except for non-editing commands such as writing
  286. the file to disc, most commands make some change
  287. to the text in dot and leave dot set to the text resulting from the change.
  288. For example, the delete command,
  289. .CW d ,
  290. deletes the text in dot, replacing it by the null string and setting dot
  291. to the result.
  292. The change command,
  293. .CW c ,
  294. replaces dot by text delimited by an arbitrary punctuation character,
  295. conventionally
  296. a slash. Thus,
  297. .P1
  298. c/Peter/
  299. .P2
  300. replaces the text in dot by the string
  301. .CW Peter .
  302. Similarly,
  303. .P1
  304. a/Peter/
  305. .P2
  306. (append) adds the string after dot, and
  307. .P1
  308. i/Peter/
  309. .P2
  310. (insert) inserts before dot.
  311. All three leave dot set to the new text,
  312. .CW Peter .
  313. .PP
  314. Newlines are part of the syntax of commands:
  315. the newline character lexically terminates a command.
  316. Within the inserted text, however, newlines are never implicit.
  317. But since it is often convenient to insert multiple lines of text,
  318. .CW sam
  319. has a special
  320. syntax for that case:
  321. .P1
  322. a
  323. some lines of text
  324. to be inserted in the file,
  325. terminated by a period
  326. on a line by itself
  327. \&.
  328. .P2
  329. In the one-line syntax, a newline character may be specified by a C-like
  330. escape, so
  331. .P1
  332. c/\en/
  333. .P2
  334. replaces dot by a single newline character.
  335. .PP
  336. .CW Sam
  337. also has a substitute command,
  338. .CW s :
  339. .P1
  340. s/\f2expression\fP/\f2replacement\fP/
  341. .P2
  342. substitutes the replacement text for the first match, in dot,
  343. of the regular expression.
  344. Thus, if dot is the string
  345. .CW Peter ,
  346. the command
  347. .P1
  348. s/t/st/
  349. .P2
  350. changes it to
  351. .CW Pester .
  352. In general,
  353. .CW s
  354. is unnecessary, but it was inherited from
  355. .CW ed
  356. and it has some convenient variations.
  357. For instance, the replacement text may include the matched text,
  358. specified by
  359. .CW & :
  360. .P1
  361. s/Peter/Oh, &, &, &, &!/
  362. .P2
  363. .PP
  364. There are also three commands that apply programs
  365. to text:
  366. .P1
  367. < \f2UNIX program\fP
  368. .P2
  369. replaces dot by the output of the UNIX program.
  370. Similarly, the
  371. .CW >
  372. command
  373. runs the program with dot as its standard input, and
  374. .CW |
  375. does both. For example,
  376. .P1
  377. | sort
  378. .P2
  379. replaces dot by the result of applying the standard sorting utility to it.
  380. Again, newlines have no special significance for these
  381. .CW sam
  382. commands.
  383. The text acted upon and resulting from these commands is not necessarily
  384. bounded by newlines, although for connection with UNIX programs,
  385. newlines may be necessary to obey conventions.
  386. .PP
  387. One more command:
  388. .CW p
  389. prints the contents of dot.
  390. Table I summarizes
  391. .CW sam 's
  392. commands.
  393. .KF
  394. .TS
  395. center;
  396. c s
  397. lfCW l.
  398. Table I. \f(CWSam\fP commands
  399. .sp .4
  400. .ft CW
  401. _
  402. .ft
  403. .sp .4
  404. \f1Text commands\fP
  405. .sp .4
  406. _
  407. .sp .4
  408. a/\f2text\fP/ Append text after dot
  409. c/\f2text\fP/ Change text in dot
  410. i/\f2text\fP/ Insert text before dot
  411. d Delete text in dot
  412. s/\f2regexp\fP/\f2text\fP/ Substitute text for match of regular expression in dot
  413. m \f2address\fP Move text in dot after address
  414. t \f2address\fP Copy text in dot after address
  415. .sp .4
  416. _
  417. .sp .4
  418. \f1Display commands\fP
  419. .sp .4
  420. _
  421. .sp .2
  422. p Print contents of dot
  423. \&= Print value (line numbers and character numbers) of dot
  424. .sp .4
  425. _
  426. .sp .4
  427. \f1File commands\fP
  428. .sp .4
  429. _
  430. .sp .2
  431. b \f2file-list\fP Set current file to first file in list that \f(CWsam\fP has in menu
  432. B \f2file-list\fP Same as \f(CWb\fP, but load new files
  433. n Print menu lines of all files
  434. D \f2file-list\fP Delete named files from \f(CWsam\fP
  435. .sp .4
  436. _
  437. .sp .4
  438. \f1I/O commands\fP
  439. .sp .4
  440. _
  441. .sp .2
  442. e \f2filename\fP Replace file with named disc file
  443. r \f2filename\fP Replace dot by contents of named disc file
  444. w \f2filename\fP Write file to named disc file
  445. f \f2filename\fP Set file name and print new menu line
  446. < \f2UNIX-command\fP Replace dot by standard output of command
  447. > \f2UNIX-command\fP Send dot to standard input of command
  448. | \f2UNIX-command\fP Replace dot by result of command applied to dot
  449. ! \f2UNIX-command\fP Run the command
  450. .sp .4
  451. _
  452. .sp .4
  453. \f1Loops and conditionals\fP
  454. .sp .4
  455. _
  456. .sp .2
  457. x/\f2regexp\fP/ \f2command\fP For each match of regexp, set dot and run command
  458. y/\f2regexp\fP/ \f2command\fP Between adjacent matches of regexp, set dot and run command
  459. X/\f2regexp\fP/ \f2command\fP Run command in each file whose menu line matches regexp
  460. Y/\f2regexp\fP/ \f2command\fP Run command in each file whose menu line does not match
  461. g/\f2regexp\fP/ \f2command\fP If dot contains a match of regexp, run command
  462. v/\f2regexp\fP/ \f2command\fP If dot does not contain a match of regexp, run command
  463. .sp .4
  464. _
  465. .sp .4
  466. \f1Miscellany\fP
  467. .sp .4
  468. _
  469. .sp .2
  470. k Set address mark to value of dot
  471. q Quit
  472. u \f2n\fP Undo last \f2n\fP (default 1) changes
  473. { } Braces group commands
  474. .sp .3
  475. .ft CW
  476. _
  477. .ft
  478. .TE
  479. .sp
  480. .KE
  481. .PP
  482. The value of dot may be changed by
  483. specifying an
  484. .I address
  485. for the command.
  486. The simplest address is a line number:
  487. .P1
  488. 3
  489. .P2
  490. refers to the third line of the file, so
  491. .P1
  492. 3d
  493. .P2
  494. deletes the third line of the file, and implicitly renumbers
  495. the lines so the old line 4 is now numbered 3.
  496. (This is one of the few places where
  497. .CW sam
  498. deals with lines directly.)
  499. Line
  500. .CW 0
  501. is the null string at the beginning of the file.
  502. If a command consists of only an address, a
  503. .CW p
  504. command is assumed, so typing an unadorned
  505. .CW 3
  506. prints line 3 on the terminal.
  507. There are a couple of other basic addresses:
  508. a period addresses dot itself; and
  509. a dollar sign
  510. .CW $ ) (
  511. addresses the null string at the end of the file.
  512. .PP
  513. An address is always a single substring of the file.
  514. Thus, the address
  515. .CW 3
  516. addresses the characters
  517. after the second newline of
  518. the file through the third newline of the file.
  519. A
  520. .I
  521. compound address
  522. .R
  523. is constructed by the comma operator
  524. .P1
  525. \f2address1\fP,\f2address2\fP
  526. .P2
  527. and addresses the substring of the file from the beginning of
  528. .I address1
  529. to the end of
  530. .I address2 .
  531. For example, the command
  532. .CW 3,5p
  533. prints the third through fifth lines of the file and
  534. .CW .,$d
  535. deletes the text from the beginning of dot to the end of the file.
  536. .PP
  537. These addresses are all absolute positions in the file, but
  538. .CW sam
  539. also has relative addresses, indicated by
  540. .CW +
  541. or
  542. .CW - .
  543. For example,
  544. .P1
  545. $-3
  546. .P2
  547. is the third line before the end of the file and
  548. .P1
  549. \&.+1
  550. .P2
  551. is the line after dot.
  552. If no address appears to the left of the
  553. .CW +
  554. or
  555. .CW - ,
  556. dot is assumed;
  557. if nothing appears to the right,
  558. .CW 1
  559. is assumed.
  560. Therefore,
  561. .CW .+1
  562. may be abbreviated to just a plus sign.
  563. .PP
  564. The
  565. .CW +
  566. operator acts relative to the end of its first argument, while the
  567. .CW -
  568. operator acts relative to the beginning. Thus
  569. .CW .+1
  570. addresses the first line after dot,
  571. .CW .-
  572. addresses the first line before dot, and
  573. .CW +-
  574. refers to the line containing the end of dot. (Dot may span multiple lines, and
  575. .CW +
  576. selects the line after the end of dot, then
  577. .CW -
  578. backs up one line.)
  579. .PP
  580. The final type of address is a regular expression, which addresses the
  581. text matched by the expression. The expression is enclosed in slashes, as in
  582. .P1
  583. /\f2expression\fP/
  584. .P2
  585. The expressions are the same as those in the UNIX program
  586. .CW egrep ,\u\s-4\&6,7\s+4\d
  587. and include closures, alternations, and so on.
  588. They find the
  589. .I
  590. leftmost longest
  591. .R
  592. string that matches the expression, that is,
  593. the first match after the point where the search is started,
  594. and if more than one match begins at the same spot, the longest such match.
  595. (I assume familiarity with the syntax for regular expressions in UNIX programs.\u\s-4\&9\s+4\d)
  596. For example,
  597. .P1
  598. /x/
  599. .P2
  600. matches the next
  601. .CW x
  602. character in the file,
  603. .P1
  604. /xx*/
  605. .P2
  606. matches the next run of one or more
  607. .CW x 's,
  608. and
  609. .P1
  610. /x|Peter/
  611. .P2
  612. matches the next
  613. .CW x
  614. or
  615. .CW Peter .
  616. For compatibility with other UNIX programs, the `any character' operator,
  617. a period,
  618. does not match a newline, so
  619. .P1
  620. /.*/
  621. .P2
  622. matches the text from dot to the end of the line, but excludes the newline
  623. and so will not match across
  624. the line boundary.
  625. .PP
  626. Regular expressions are always relative addresses.
  627. The direction is forwards by default,
  628. so
  629. .CW /Peter/
  630. is really an abbreviation for
  631. .CW +/Peter/ .
  632. The search can be reversed with a minus sign, so
  633. .P1
  634. .CW -/Peter/
  635. .P2
  636. finds the first
  637. .CW Peter
  638. before dot.
  639. Regular expressions may be used with other address forms, so
  640. .CW 0+/Peter/
  641. finds the first
  642. .CW Peter
  643. in the file and
  644. .CW $-/Peter/
  645. finds the last.
  646. Table II summarizes
  647. .CW sam 's
  648. addresses.
  649. .KF
  650. .TS
  651. center;
  652. c s
  653. lfCW l.
  654. Table II. \f(CWSam\fP addresses
  655. .sp .4
  656. .ft CW
  657. _
  658. .ft
  659. .sp .4
  660. \f1Simple addresses\fP
  661. .sp .4
  662. _
  663. .sp .2
  664. #\f2n\fP The empty string after character \f2n\fP
  665. \f2n\fP Line \f2n\fP.
  666. /\f2regexp\fP/ The first following match of the regular expression
  667. -/\f2regexp\fP/ The first previous match of the regular expression
  668. $ The null string at the end of the file
  669. \&. Dot
  670. \&' The address mark, set by \f(CWk\fP command
  671. "\f2regexp\fP" Dot in the file whose menu line matches regexp
  672. .sp .4
  673. _
  674. .sp .4
  675. \f1Compound addresses\fP
  676. .sp .4
  677. _
  678. .sp .2
  679. \f2a1\fP+\f2a2\fP The address \f2a2\fP evaluated starting at right of \f2a1\fP
  680. \f2a1\fP-\f2a2\fP \f2a2\fP evaluated in the reverse direction starting at left of \f2a1\fP
  681. \f2a1\fP,\f2a2\fP From the left of \f2a1\fP to the right of \f2a2\fP (default \f(CW0,$\fP)
  682. \f2a1\fP;\f2a2\fP Like \f(CW,\fP but sets dot after evaluating \f2a1\fP
  683. .sp .4
  684. _
  685. .sp .4
  686. .T&
  687. c s.
  688. T{
  689. The operators
  690. .CW +
  691. and
  692. .CW -
  693. are high precedence, while
  694. .CW ,
  695. and
  696. .CW ;
  697. are low precedence.
  698. In both
  699. .CW +
  700. and
  701. .CW -
  702. forms,
  703. .I a2
  704. defaults to 1 and
  705. .I a1
  706. defaults to dot.
  707. If both
  708. .I a1
  709. and
  710. .I a2
  711. are present,
  712. .CW +
  713. may be elided.
  714. T}
  715. .sp .5
  716. .ft CW
  717. _
  718. .ft
  719. .TE
  720. .sp
  721. .KE
  722. .PP
  723. The language discussed so far will not seem novel
  724. to people who use UNIX text editors
  725. such as
  726. .CW ed
  727. or
  728. .CW vi .\u\s-4\&9\s+4\d
  729. Moreover, the kinds of editing operations these commands allow, with the exception
  730. of regular expressions and line numbers,
  731. are clearly more conveniently handled by a mouse-based interface.
  732. Indeed,
  733. .CW sam 's
  734. mouse language (discussed at length below) is the means by which
  735. simple changes are usually made.
  736. For large or repetitive changes, however, a textual language
  737. outperforms a manual interface.
  738. .PP
  739. Imagine that, instead of deleting just one occurrence of the string
  740. .CW Peter ,
  741. we wanted to eliminate every
  742. .CW Peter .
  743. What's needed is an iterator that runs a command for each occurrence of some
  744. text.
  745. .CW Sam 's
  746. iterator is called
  747. .CW x ,
  748. for extract:
  749. .P1
  750. x/\f2expression\fP/ \f2command\fP
  751. .P2
  752. finds all matches in dot of the specified expression, and for each
  753. such match, sets dot to the text matched and runs the command.
  754. So to delete all the
  755. .CW Peters:
  756. .P1
  757. 0,$ x/Peter/ d
  758. .P2
  759. (Blanks in these examples are to improve readability;
  760. .CW sam
  761. neither requires nor interprets them.)
  762. This searches the entire file
  763. .CW 0,$ ) (
  764. for occurrences of the string
  765. .CW Peter ,
  766. and runs the
  767. .CW d
  768. command with dot set to each such occurrence.
  769. (By contrast, the comparable
  770. .CW ed
  771. command would delete all
  772. .I lines
  773. containing
  774. .CW Peter ;
  775. .CW sam
  776. deletes only the
  777. .CW Peters .)
  778. The address
  779. .CW 0,$
  780. is commonly used, and may be abbreviated to just a comma.
  781. As another example,
  782. .P1
  783. , x/Peter/ p
  784. .P2
  785. prints a list of
  786. .CW Peters,
  787. one for each appearance in the file, with no intervening text (not even newlines
  788. to separate the instances).
  789. .PP
  790. Of course, the text extracted by
  791. .CW x
  792. may be selected by a regular expression,
  793. which complicates deciding what set of matches is chosen \(em
  794. matches may overlap. This is resolved by generating the matches
  795. starting from the beginning of dot using the leftmost-longest rule,
  796. and searching for each match starting from the end of the previous one.
  797. Regular expressions may also match null strings, but a null match
  798. adjacent to a non-null match is never selected; at least one character
  799. must intervene.
  800. For example,
  801. .P1
  802. , c/AAA/
  803. x/B*/ c/-/
  804. , p
  805. .P2
  806. produces as output
  807. .P1
  808. -A-A-A-
  809. .P2
  810. because the pattern
  811. .CW B*
  812. matches the null strings separating the
  813. .CW A 's.
  814. .PP
  815. The
  816. .CW x
  817. command has a complement,
  818. .CW y ,
  819. with similar syntax, that executes the command with dot set to the text
  820. .I between
  821. the matches of the expression.
  822. For example,
  823. .P1
  824. , c/AAA/
  825. y/A/ c/-/
  826. , p
  827. .P2
  828. produces the same result as the example above.
  829. .PP
  830. The
  831. .CW x
  832. and
  833. .CW y
  834. commands are looping constructs, and
  835. .CW sam
  836. has a pair of conditional commands to go with them.
  837. They have similar syntax:
  838. .P1
  839. g/\f2expression\fP/ \f2command\fP
  840. .P2
  841. (guard)
  842. runs the command exactly once if dot contains a match of the expression.
  843. This is different from
  844. .CW x ,
  845. which runs the command for
  846. .I each
  847. match:
  848. .CW x
  849. loops;
  850. .CW g
  851. merely tests, without changing the value of dot.
  852. Thus,
  853. .P1
  854. , x/Peter/ d
  855. .P2
  856. deletes all occurrences of
  857. .CW Peter ,
  858. but
  859. .P1
  860. , g/Peter/ d
  861. .P2
  862. deletes the whole file (reduces it to a null string) if
  863. .CW Peter
  864. occurs anywhere in the text.
  865. The complementary conditional is
  866. .CW v ,
  867. which runs the command if there is
  868. .I no
  869. match of the expression.
  870. .PP
  871. These control-structure-like commands may be composed to construct more
  872. involved operations. For example, to print those lines of text that
  873. contain the string
  874. .CW Peter :
  875. .P1
  876. , x/.*\en/ g/Peter/ p
  877. .P2
  878. The
  879. .CW x
  880. breaks the file into lines, the
  881. .CW g
  882. selects those lines containing
  883. .CW Peter ,
  884. and the
  885. .CW p
  886. prints them.
  887. This command gives an address for the
  888. .CW x
  889. command (the whole file), but because
  890. .CW g
  891. does not have an explicit address, it applies to the value of
  892. dot produced by the
  893. .CW x
  894. command, that is, to each line.
  895. All commands in
  896. .CW sam
  897. except for the command to write a file to disc use dot for the
  898. default address.
  899. .PP
  900. Composition may be continued indefinitely.
  901. .P1
  902. , x/.*\en/ g/Peter/ v/SaltPeter/ p
  903. .P2
  904. prints those lines containing
  905. .CW Peter
  906. but
  907. .I not
  908. those containing
  909. .CW SaltPeter .
  910. .SH 2
  911. Structural Regular Expressions
  912. .LP
  913. Unlike other UNIX text editors,
  914. including the non-interactive ones such as
  915. .CW sed
  916. and
  917. .CW awk ,\u\s-4\&7\s+4\d
  918. .CW sam
  919. is good for manipulating files with multi-line `records.'
  920. An example is an on-line phone book composed of records,
  921. separated by blank lines, of the form
  922. .P1
  923. Herbert Tic
  924. 44 Turnip Ave., Endive, NJ
  925. 201-5555642
  926. Norbert Twinge
  927. 16 Potato St., Cabbagetown, NJ
  928. 201-5553145
  929. \&...
  930. .P2
  931. The format may be encoded as a regular expression:
  932. .P1
  933. (.+\en)+
  934. .P2
  935. that is, a sequence of one or more non-blank lines.
  936. The command to print Mr. Tic's entire record is then
  937. .P1
  938. , x/(.+\en)+/ g/^Herbert Tic$/ p
  939. .P2
  940. and that to extract just the phone number is
  941. .P1
  942. , x/(.+\en)+/ g/^Herbert Tic$/ x/^[0-9]*-[0-9]*\en/ p
  943. .P2
  944. The latter command breaks the file into records,
  945. chooses Mr. Tic's record,
  946. extracts the phone number from the record,
  947. and finally prints the number.
  948. .PP
  949. A more involved problem is that of
  950. renaming a particular variable, say
  951. .CW n ,
  952. to
  953. .CW num
  954. in a C program.
  955. The obvious first attempt,
  956. .P1
  957. , x/n/ c/num/
  958. .P2
  959. is badly flawed: it changes not only the variable
  960. .CW n
  961. but any letter
  962. .CW n
  963. that appears.
  964. We need to extract all the variables, and select those that match
  965. .CW n
  966. and only
  967. .CW n :
  968. .P1
  969. , x/[A-Za-z_][A-Za-z_0-9]*/ g/n/ v/../ c/num/
  970. .P2
  971. The pattern
  972. .CW [A-Za-z_][A-Za-z_0-9]*
  973. matches C identifiers.
  974. Next
  975. .CW g/n/
  976. selects those containing an
  977. .CW n .
  978. Then
  979. .CW v/../
  980. rejects those containing two (or more) characters, and finally
  981. .CW c/num/
  982. changes the remainder (identifiers
  983. .CW n )
  984. to
  985. .CW num .
  986. This version clearly works much better, but there may still be problems.
  987. For example, in C character and string constants, the sequence
  988. .CW \en
  989. is interpreted as a newline character, and we don't want to change it to
  990. .CW \enum.
  991. This problem can be forestalled with a
  992. .CW y
  993. command:
  994. .P1
  995. , y/\e\en/ x/[A-Za-z_][A-Za-z_0-9]*/ g/n/ v/../ c/num/
  996. .P2
  997. (the second
  998. .CW \e
  999. is necessary because of lexical conventions in regular expressions),
  1000. or we could even reject character constants and strings outright:
  1001. .P1 0
  1002. ,y/'[^']*'/ y/"[^"]*"/ x/[A-Za-z_][A-Za-z_0-9]*/ g/n/ v/../ c/num/
  1003. .P2
  1004. The
  1005. .CW y
  1006. commands in this version exclude from consideration all character constants
  1007. and strings.
  1008. The only remaining problem is to deal with the possible occurrence of
  1009. .CW \e'
  1010. or
  1011. .CW \e"
  1012. within these sequences, but it's easy to see how to resolve this difficulty.
  1013. .PP
  1014. The point of these composed commands is successive refinement.
  1015. A simple version of the command is tried, and if it's not good enough,
  1016. it can be honed by adding a clause or two.
  1017. (Mistakes can be undone; see below.
  1018. Also, the mouse language makes it unnecessary to retype the command each time.)
  1019. The resulting chains of commands are somewhat reminiscent of
  1020. shell pipelines.\u\s-4\&7\s+4\d
  1021. Unlike pipelines, though, which pass along modified
  1022. .I data ,
  1023. .CW sam
  1024. commands pass a
  1025. .I view
  1026. of the data.
  1027. The text at each step of the command is the same, but which pieces
  1028. are selected is refined step by step until the correct piece is
  1029. available to the final step of the command line, which ultimately makes the change.
  1030. .PP
  1031. In other UNIX programs, regular expressions are used only for selection,
  1032. as in the
  1033. .CW sam
  1034. .CW g
  1035. command, never for extraction as in the
  1036. .CW x
  1037. or
  1038. .CW y
  1039. command.
  1040. For example, patterns in
  1041. .CW awk \u\s-4\&7\s+4\d
  1042. are used to select lines to be operated on, but cannot be used
  1043. to describe the format of the input text, or to handle newline-free text.
  1044. The use of regular expressions to describe the structure of a piece
  1045. of text rather than its contents, as in the
  1046. .CW x
  1047. command,
  1048. has been given a name:
  1049. .I
  1050. structural regular expressions.
  1051. .R
  1052. When they are composed, as in the above example,
  1053. they are pleasantly expressive.
  1054. Their use is discussed at greater length elsewhere.\u\s-4\&10\s+4\d
  1055. .PP
  1056. .SH 2
  1057. Multiple files
  1058. .LP
  1059. .CW Sam
  1060. has a few other commands, mostly relating to input and output.
  1061. .P1
  1062. e discfilename
  1063. .P2
  1064. replaces the contents and name of the current file with those of the named
  1065. disc file;
  1066. .P1
  1067. w discfilename
  1068. .P2
  1069. writes the contents to the named disc file; and
  1070. .P1
  1071. r discfilename
  1072. .P2
  1073. replaces dot with the contents of the named disc file.
  1074. All these commands use the current file's name if none is specified.
  1075. Finally,
  1076. .P1
  1077. f discfilename
  1078. .P2
  1079. changes the name associated with the file and displays the result:
  1080. .P1
  1081. \&'-. discfilename
  1082. .P2
  1083. This output is called the file's
  1084. .I
  1085. menu line,
  1086. .R
  1087. because it is the contents of the file's line in the button 3 menu (described
  1088. in the
  1089. next section).
  1090. The first three characters are a concise notation for the state of the file.
  1091. The apostrophe signifies that the file is modified.
  1092. The minus sign indicates the number of windows
  1093. open on the file (see the next section):
  1094. .CW -
  1095. means none,
  1096. .CW +
  1097. means one, and
  1098. .CW *
  1099. means more than one.
  1100. Finally, the period indicates that this is the current file.
  1101. These characters are useful for controlling the
  1102. .CW X
  1103. command, described shortly.
  1104. .PP
  1105. .CW Sam
  1106. may be started with a set of disc files (such as all the source for
  1107. a program) by invoking it with a list of file names as arguments, and
  1108. more may be added or deleted on demand.
  1109. .P1
  1110. B discfile1 discfile2 ...
  1111. .P2
  1112. adds the named files to
  1113. .CW sam 's
  1114. list, and
  1115. .P1
  1116. D discfile1 discfile2 ...
  1117. .P2
  1118. removes them from
  1119. .CW sam 's
  1120. memory (without effect on associated disc files).
  1121. Both these commands have a syntax for using the shell\u\s-4\&7\s+4\d
  1122. (the UNIX command interpreter) to generate the lists:
  1123. .P1
  1124. B <echo *.c
  1125. .P2
  1126. will add all C source files, and
  1127. .P1
  1128. B <grep -l variable *.c
  1129. .P2
  1130. will add all C source files referencing a particular variable
  1131. (the UNIX command
  1132. .CW grep\ -l
  1133. lists all files in its arguments that contain matches of
  1134. the specified regular expression).
  1135. Finally,
  1136. .CW D
  1137. without arguments deletes the current file.
  1138. .PP
  1139. There are two ways to change which file is current:
  1140. .P1
  1141. b filename
  1142. .P2
  1143. makes the named file current.
  1144. The
  1145. .CW B
  1146. command
  1147. does the same, but also adds any new files to
  1148. .CW sam 's
  1149. list.
  1150. (In practice, of course, the current file
  1151. is usually chosen by mouse actions, not by textual commands.)
  1152. The other way is to use a form of address that refers to files:
  1153. .P1
  1154. "\f2expression\fP" \f2address\fP
  1155. .P2
  1156. refers to the address evaluated in the file whose menu line
  1157. matches the expression (there must be exactly one match).
  1158. For example,
  1159. .P1
  1160. "peter.c" 3
  1161. .P2
  1162. refers to the third line of the file whose name matches
  1163. .CW peter.c .
  1164. This is most useful in the move
  1165. .CW m ) (
  1166. and copy
  1167. .CW t ) (
  1168. commands:
  1169. .P1
  1170. 0,$ t "peter.c" 0
  1171. .P2
  1172. makes a copy of the current file at the beginning of
  1173. .CW peter.c .
  1174. .PP
  1175. The
  1176. .CW X
  1177. command
  1178. is a looping construct, like
  1179. .CW x ,
  1180. that refers to files instead of strings:
  1181. .P1
  1182. X/\f2expression\fP/ \f2command\fP
  1183. .P2
  1184. runs the command in all
  1185. files whose menu lines match the expression. The best example is
  1186. .P1
  1187. X/'/ w
  1188. .P2
  1189. which writes to disc all modified files.
  1190. .CW Y
  1191. is the complement of
  1192. .CW X :
  1193. it runs the command on all files whose menu lines don't match the expression:
  1194. .P1
  1195. Y/\e.c/ D
  1196. .P2
  1197. deletes all files that don't have
  1198. .CW \&.c
  1199. in their names, that is, it keeps all C source files and deletes the rest.
  1200. .PP
  1201. Braces allow commands to be grouped, so
  1202. .P1
  1203. {
  1204. \f2command1\fP
  1205. \f2command2\fP
  1206. }
  1207. .P2
  1208. is syntactically a single command that runs two commands.
  1209. Thus,
  1210. .P1
  1211. X/\e.c/ ,g/variable/ {
  1212. f
  1213. , x/.*\en/ g/variable/ p
  1214. }
  1215. .P2
  1216. finds all occurrences of
  1217. .CW variable
  1218. in C source files, and prints
  1219. out the file names and lines of each match.
  1220. The precise semantics of compound operations is discussed in the implementation
  1221. sections below.
  1222. .PP
  1223. Finally,
  1224. the undo command,
  1225. .CW u ,
  1226. undoes the last command,
  1227. no matter how many files were affected.
  1228. Multiple undo operations move further back in time, so
  1229. .P1
  1230. u
  1231. u
  1232. .P2
  1233. (which may be abbreviated
  1234. .CW u2 )
  1235. undoes the last two commands. An undo may not be undone, however, nor
  1236. may any command that adds or deletes files.
  1237. Everything else is undoable, though, including for example
  1238. .CW e
  1239. commands:
  1240. .P1
  1241. e filename
  1242. u
  1243. .P2
  1244. restores the state of the file completely, including its name, dot,
  1245. and modified bit. Because of the undo, potentially dangerous commands
  1246. are not guarded by confirmations. Only
  1247. .CW D ,
  1248. which destroys the information necessary to restore itself, is protected.
  1249. It will not delete a modified file, but a second
  1250. .CW D
  1251. of the same file will succeed regardless.
  1252. The
  1253. .CW q
  1254. command, which exits
  1255. .CW sam ,
  1256. is similarly guarded.
  1257. .SH 2
  1258. Mouse Interface
  1259. .LP
  1260. .CW Sam
  1261. is most commonly run
  1262. connected to a bitmap display and mouse for interactive editing.
  1263. The only difference in the command language
  1264. between regular, mouse-driven
  1265. .CW sam
  1266. and
  1267. .CW sam\ -d
  1268. is that if an address
  1269. is provided without a command,
  1270. .CW sam\ -d
  1271. will print the text referenced by the address, but
  1272. regular
  1273. .CW sam
  1274. will highlight it on the screen \(em in fact,
  1275. dot is always highlighted (see Figure 2).
  1276. .WS 1
  1277. .KF
  1278. .XP fig3 2.04i
  1279. .Cs
  1280. Figure 2. A
  1281. .CW sam
  1282. window. The scroll bar down the left
  1283. represents the file, with the bubble showing the fraction
  1284. visible in the window.
  1285. The scroll bar may be manipulated by the mouse for convenient browsing.
  1286. The current text,
  1287. which is highlighted, need not fit on a line. Here it consists of one partial
  1288. line, one complete line, and final partial line.
  1289. .Ce
  1290. .KE
  1291. .PP
  1292. Each file may have zero or more windows open on the display.
  1293. At any time, only one window in all of
  1294. .CW sam
  1295. is the
  1296. .I
  1297. current window,
  1298. .R
  1299. that is, the window to which typing and mouse actions refer;
  1300. this may be the
  1301. .CW sam
  1302. window (that in which commands may be typed)
  1303. or one of the file windows.
  1304. When a file has multiple windows, the image of the file in each window
  1305. is always kept up to date.
  1306. The current file is the last file affected by a command,
  1307. so if the
  1308. .CW sam
  1309. window is current,
  1310. the current window is not a window on the current file.
  1311. However, each window on a file has its own value of dot,
  1312. and when switching between windows on a single file,
  1313. the file's value of dot is changed to that of the window.
  1314. Thus, flipping between windows behaves in the obvious, convenient way.
  1315. .PP
  1316. The mouse on the Blit has three buttons, numbered left to right.
  1317. Button 3 has a list of commands to manipulate windows,
  1318. followed by a list of `menu lines' exactly as printed by the
  1319. .CW f
  1320. command, one per file (not one per window).
  1321. These menu lines are sorted by file name.
  1322. If the list is long, the Blit menu software will make it more manageable
  1323. by generating a scrolling menu instead of an unwieldy long list.
  1324. Using the menu to select a file from the list makes that file the current
  1325. file, and the most recently current window in that file the current window.
  1326. But if that file is already current, selecting it in the menu cycles through
  1327. the windows on the file; this simple trick avoids a special menu to
  1328. choose windows on a file.
  1329. If there is no window open on the file,
  1330. .CW sam
  1331. changes the mouse cursor to prompt the user to create one.
  1332. .PP
  1333. The commands on the button 3 menu are straightforward (see Figure 3), and
  1334. are like the commands to manipulate windows in
  1335. .CW mux ,\u\s-4\&8\s+4\d
  1336. the Blit's window system.
  1337. .CW New
  1338. makes a new file, and gives it one empty window, whose size is determined
  1339. by a rectangle swept by the mouse.
  1340. .CW Zerox
  1341. prompts for a window to be selected, and
  1342. makes a clone of that window; this is how multiple windows are created on one file.
  1343. .CW Reshape
  1344. changes the size of the indicated window, and
  1345. .CW close
  1346. deletes it. If that is the last window open on the file,
  1347. .CW close
  1348. first does a
  1349. .CW D
  1350. command on the file.
  1351. .CW Write
  1352. is identical to a
  1353. .CW w
  1354. command on the file; it is in the menu purely for convenience.
  1355. Finally,
  1356. .CW ~~sam~~
  1357. is a menu item that appears between the commands and the file names.
  1358. Selecting it makes the
  1359. .CW sam
  1360. window the current window,
  1361. causing subsequent typing to be interpreted as commands.
  1362. .KF
  1363. .XP fig2 2.74i
  1364. .Cs
  1365. Figure 3. The menu on button 3.
  1366. The black rectangle on the left is a scroll bar; the menu is limited to
  1367. the length shown to prevent its becoming unwieldy.
  1368. Above the
  1369. .CW ~~sam~~
  1370. line is a list of commands;
  1371. beneath it is a list of files, presented exactly as with the
  1372. .CW f
  1373. command.
  1374. .Ce
  1375. .KE
  1376. .PP
  1377. When
  1378. .CW sam
  1379. requests that a window be swept, in response to
  1380. .CW new ,
  1381. .CW zerox
  1382. or
  1383. .CW reshape ,
  1384. it changes the mouse cursor from the usual arrow to a box with
  1385. a small arrow.
  1386. In this state, the mouse may be used to indicate an arbitrary rectangle by
  1387. pressing button 3 at one corner and releasing it at the opposite corner.
  1388. More conveniently,
  1389. button 3 may simply be clicked,
  1390. whereupon
  1391. .CW sam
  1392. creates the maximal rectangle that contains the cursor
  1393. and abuts the
  1394. .CW sam
  1395. window.
  1396. By placing the
  1397. .CW sam
  1398. window in the middle of the screen, the user can define two regions (one above,
  1399. one below) in which stacked fully-overlapping
  1400. windows can be created with minimal fuss (see Figure 1).
  1401. This simple user interface trick makes window creation noticeably easier.
  1402. .PP
  1403. The cut-and-paste editor is essentially the same as that in Smalltalk-80.\u\s-4\&11\s+4\d
  1404. The text in dot is always highlighted on the screen.
  1405. When a character is typed it replaces dot, and sets dot to the null
  1406. string after the character. Thus, ordinary typing inserts text.
  1407. Button 1 is used for selection:
  1408. pressing the button, moving the mouse, and lifting the button
  1409. selects (sets dot to) the text between the points where the
  1410. button was pressed and released.
  1411. Pressing and releasing at the same point selects a null string; this
  1412. is called clicking. Clicking twice quickly, or
  1413. .I
  1414. double clicking,
  1415. .R
  1416. selects larger objects;
  1417. for example, double clicking in a word selects the word,
  1418. double clicking just inside an opening bracket selects the text
  1419. contained in the brackets (handling nested brackets correctly),
  1420. and similarly for
  1421. parentheses, quotes, and so on.
  1422. The double-clicking rules reflect a bias toward
  1423. programmers.
  1424. If
  1425. .CW sam
  1426. were intended more for word processing, double-clicks would probably
  1427. select linguistic structures such as sentences.
  1428. .PP
  1429. If button 1 is pressed outside the current window, it makes the indicated
  1430. window current.
  1431. This is the easiest way to switch between windows and files.
  1432. .PP
  1433. Pressing button 2 brings up a menu of editing functions (see Figure 4).
  1434. These mostly apply to the selected text:
  1435. .CW cut
  1436. deletes the selected text, and remembers it in a hidden buffer called the
  1437. .I
  1438. snarf buffer,
  1439. .R
  1440. .CW paste
  1441. replaces the selected text by the contents of the snarf buffer,
  1442. .CW snarf
  1443. just copies the selected text to the snarf buffer,
  1444. .CW look
  1445. searches forward for the next literal occurrence of the selected text, and
  1446. .CW <mux>
  1447. exchanges snarf buffers with the window system in which
  1448. .CW sam
  1449. is running.
  1450. Finally, the last regular expression used appears as a menu entry
  1451. to search
  1452. forward for the next occurrence of a match for the expression.
  1453. .WS 1
  1454. .KF
  1455. .XP fig4 1.20i
  1456. .Cs
  1457. Figure 4. The menu on button 2.
  1458. The bottom entry tracks the most recently used regular expression, which may
  1459. be literal text.
  1460. .Ce
  1461. .KE
  1462. .PP
  1463. The relationship between the command language and the mouse language is
  1464. entirely due to the equality of dot and the selected text chosen
  1465. with button 1 on the mouse.
  1466. For example, to make a set of changes in a C subroutine, dot can be
  1467. set by double clicking on the left brace that begins the subroutine,
  1468. which sets dot for the command language.
  1469. An address-free command then typed in the
  1470. .CW sam
  1471. window will apply only to the text between the opening and closing
  1472. braces of the function.
  1473. The idea is to select what you want, and then say what you want
  1474. to do with it, whether invoked by a menu selection or by a typed command.
  1475. And of course, the value of dot is highlighted on
  1476. the display after the command completes.
  1477. This relationship between mouse interface and command language
  1478. is clumsy to explain, but comfortable, even natural, in practice.
  1479. .SH
  1480. The Implementation
  1481. .LP
  1482. The next few sections describe how
  1483. .CW sam
  1484. is put together, first the host part,
  1485. then the inter-component communication,
  1486. then the terminal part.
  1487. After explaining how the command language is implemented,
  1488. the discussion follows (roughly) the path of a character
  1489. from the temporary file on disc to the screen.
  1490. The presentation centers on the data structures,
  1491. because that is how the program was designed and because
  1492. the algorithms are easy to provide, given the right data
  1493. structures.
  1494. .SH 2
  1495. Parsing and execution
  1496. .LP
  1497. The command language is interpreted by parsing each command with a
  1498. table-driven recursive
  1499. descent parser, and when a complete command is assembled, invoking a top-down
  1500. executor.
  1501. Most editors instead employ a simple character-at-a-time
  1502. lexical scanner.
  1503. Use of a parser makes it
  1504. easy and unambiguous to detect when a command is complete,
  1505. which has two advantages.
  1506. First, escape conventions such as backslashes to quote
  1507. multiple-line commands are unnecessary; if the command isn't finished,
  1508. the parser keeps reading. For example, a multiple-line append driven by an
  1509. .CW x
  1510. command is straightforward:
  1511. .P1
  1512. x/.*\en/ g/Peter/ a
  1513. one line about Peter
  1514. another line about Peter
  1515. \&.
  1516. .P2
  1517. Other UNIX editors would require a backslash after all but the last line.
  1518. .PP
  1519. The other advantage is specific to the two-process structure of
  1520. .CW sam .
  1521. The host process must decide when a command is completed so the
  1522. command interpreter can be called. This problem is easily resolved
  1523. by having the lexical analyzer read the single stream of events from the
  1524. terminal, directly executing all typing and mouse commands,
  1525. but passing to the parser characters typed to the
  1526. .CW sam
  1527. command window.
  1528. This scheme is slightly complicated by the availability of cut-and-paste
  1529. editing in the
  1530. .CW sam
  1531. window, but that difficulty is resolved by applying the rules
  1532. used in
  1533. .CW mux :
  1534. when a newline is typed to the
  1535. .CW sam
  1536. window, all text between the newline and the previously typed newline
  1537. is made available to the parser.
  1538. This permits arbitrary editing to be done to a command before
  1539. typing newline and thereby requesting execution.
  1540. .PP
  1541. The parser is driven by a table because the syntax of addresses
  1542. and commands is regular enough
  1543. to be encoded compactly. There are few special cases, such as the
  1544. replacement text in a substitution, so the syntax of almost all commands
  1545. can be encoded with a few flags.
  1546. These include whether the command allows an address (for example,
  1547. .CW e
  1548. does not), whether it takes a regular expression (as in
  1549. .CW x
  1550. and
  1551. .CW s ),
  1552. whether it takes replacement text (as in
  1553. .CW c
  1554. or
  1555. .CW i ),
  1556. which may be multi-line, and so on.
  1557. The internal syntax of regular expressions is handled by a separate
  1558. parser; a regular expression is a leaf of the command parse tree.
  1559. Regular expressions are discussed fully in the next section.
  1560. .PP
  1561. The parser table also has information about defaults, so the interpreter
  1562. is always called with a complete tree. For example, the parser fills in
  1563. the implicit
  1564. .CW 0
  1565. and
  1566. .CW $
  1567. in the abbreviated address
  1568. .CW ,
  1569. (comma),
  1570. inserts a
  1571. .CW +
  1572. to the left of an unadorned regular expression in an address,
  1573. and provides the usual default address
  1574. .CW .
  1575. (dot) for commands that expect an address but are not given one.
  1576. .PP
  1577. Once a complete command is parsed, the evaluation is easy.
  1578. The address is evaluated left-to-right starting from the value of dot,
  1579. with a mostly ordinary expression evaluator.
  1580. Addresses, like many of the data structures in
  1581. .CW sam ,
  1582. are held in a C structure and passed around by value:
  1583. .P1
  1584. typedef long Posn; /* Position in a file */
  1585. typedef struct Range{
  1586. Posn p1, p2;
  1587. }Range;
  1588. typedef struct Address{
  1589. Range r;
  1590. File *f;
  1591. }Address;
  1592. .P2
  1593. An address is encoded as a substring (character positions
  1594. .CW p1
  1595. to
  1596. .CW p2 )
  1597. in a file
  1598. .CW f .
  1599. (The data type
  1600. .CW File
  1601. is described in detail below.)
  1602. .PP
  1603. The address interpreter is an
  1604. .CW Address -valued
  1605. function that traverses the parse tree describing an address (the
  1606. parse tree for the address has type
  1607. .CW Addrtree ):
  1608. .P1
  1609. Address
  1610. address(ap, a, sign)
  1611. Addrtree *ap;
  1612. Address a;
  1613. int sign;
  1614. {
  1615. Address a2;
  1616. do
  1617. switch(ap->type){
  1618. case '.':
  1619. a=a.f->dot;
  1620. break;
  1621. case '$':
  1622. a.r.p1=a.r.p2=a.f->nbytes;
  1623. break;
  1624. case '"':
  1625. a=matchfile(a, ap->aregexp)->dot;
  1626. break;
  1627. case ',':
  1628. a2=address(ap->right, a, 0);
  1629. a=address(ap->left, a, 0);
  1630. if(a.f!=a2.f || a2.r.p2<a.r.p1)
  1631. error(Eorder);
  1632. a.r.p2=a2.r.p2;
  1633. return a;
  1634. /* and so on */
  1635. }
  1636. while((ap=ap->right)!=0);
  1637. return a;
  1638. }
  1639. .P2
  1640. .PP
  1641. Throughout, errors are handled by a non-local
  1642. .CW goto
  1643. (a
  1644. .CW setjmp/longjmp
  1645. in C terminology)
  1646. hidden in a routine called
  1647. .CW error
  1648. that immediately aborts the execution, retracts any
  1649. partially made changes (see the section below on `undoing'), and
  1650. returns to the top level of the parser.
  1651. The argument to
  1652. .CW error
  1653. is an enumeration type that
  1654. is translated to a terse but possibly helpful
  1655. message such as `?addresses out of order.'
  1656. Very common messages are kept short; for example the message for
  1657. a failed regular expression search is `?search.'
  1658. .PP
  1659. Character addresses such as
  1660. .CW #3
  1661. are trivial to implement, as the
  1662. .CW File
  1663. data structure is accessible by character number.
  1664. However,
  1665. .CW sam
  1666. keeps no information about the position of newlines \(em it is too
  1667. expensive to track dynamically \(em so line addresses are computed by reading
  1668. the file, counting newlines. Except in very large files, this has proven
  1669. acceptable: file access is fast enough to make the technique practical,
  1670. and lines are not central to the structure of the command language.
  1671. .PP
  1672. The command interpreter, called
  1673. .CW cmdexec ,
  1674. is also straightforward. The parse table includes a
  1675. function to call to interpret a particular command. That function
  1676. receives as arguments
  1677. the calculated address
  1678. for the command
  1679. and the command tree (of type
  1680. .CW Cmdtree ),
  1681. which may contain information such as the subtree for compound commands.
  1682. Here, for example, is the function for the
  1683. .CW g
  1684. and
  1685. .CW v
  1686. commands:
  1687. .P1
  1688. int
  1689. g_cmd(a, cp)
  1690. Address a;
  1691. Cmdtree *cp;
  1692. {
  1693. compile(cp->regexp);
  1694. if(execute(a.f, a.r.p1, a.r.p2)!=(cp->cmdchar=='v')){
  1695. a.f->dot=a;
  1696. return cmdexec(a, cp->subcmd);
  1697. }
  1698. return TRUE; /* cause execution to continue */
  1699. }
  1700. .P2
  1701. .CW Compile "" (
  1702. and
  1703. .CW execute
  1704. are part of the regular expression code, described in the next section.)
  1705. Because the parser and the
  1706. .CW File
  1707. data structure do most of the work, most commands
  1708. are similarly brief.
  1709. .SH 2
  1710. Regular expressions
  1711. .LP
  1712. The regular expression code in
  1713. .CW sam
  1714. is an interpreted, rather than compiled on-the-fly, implementation of Thompson's
  1715. non-deterministic finite automaton algorithm.\u\s-4\&12\s+4\d
  1716. The syntax and semantics of the expressions are as in the UNIX program
  1717. .CW egrep ,
  1718. including alternation, closures, character classes, and so on.
  1719. The only changes in the notation are two additions:
  1720. .CW \en
  1721. is translated to, and matches, a newline character, and
  1722. .CW @
  1723. matches any character. In
  1724. .CW egrep ,
  1725. the character
  1726. .CW \&.
  1727. matches any character except newline, and in
  1728. .CW sam
  1729. the same rule seemed safest, to prevent idioms like
  1730. .CW \&.*
  1731. from spanning newlines.
  1732. .CW Egrep
  1733. expressions are arguably too complicated for an interactive editor \(em
  1734. certainly it would make sense if all the special characters were two-character
  1735. sequences, so that most of the punctuation characters wouldn't have
  1736. peculiar meanings \(em but for an interesting command language, full
  1737. regular expressions are necessary, and
  1738. .CW egrep
  1739. defines the full regular expression syntax for UNIX programs.
  1740. Also, it seemed superfluous to define a new syntax, since various UNIX programs
  1741. .CW ed , (
  1742. .CW egrep
  1743. and
  1744. .CW vi )
  1745. define too many already.
  1746. .PP
  1747. The expressions are compiled by a routine,
  1748. .CW compile ,
  1749. that generates the description of the non-deterministic finite state machine.
  1750. A second routine,
  1751. .CW execute ,
  1752. interprets the machine to generate the leftmost-longest match of the
  1753. expression in a substring of the file.
  1754. The algorithm is described elsewhere.\u\s-4\&12,13\s+4\d
  1755. .CW Execute
  1756. reports
  1757. whether a match was found, and sets a global variable,
  1758. of type
  1759. .CW Range ,
  1760. to the substring matched.
  1761. .PP
  1762. A trick is required to evaluate the expression in reverse, such as when
  1763. searching backwards for an expression.
  1764. For example,
  1765. .P1
  1766. -/P.*r/
  1767. .P2
  1768. looks backwards through the file for a match of the expression.
  1769. The expression, however, is defined for a forward search.
  1770. The solution is to construct a machine identical to the machine
  1771. for a forward search except for a reversal of all the concatenation
  1772. operators (the other operators are symmetric under direction reversal),
  1773. to exchange the meaning of the operators
  1774. .CW ^
  1775. and
  1776. .CW $ ,
  1777. and then to read the file backwards, looking for the
  1778. usual earliest longest match.
  1779. .PP
  1780. .CW Execute
  1781. generates only one match each time it is called.
  1782. To interpret looping constructs such as the
  1783. .CW x
  1784. command,
  1785. .CW sam
  1786. must therefore synchronize between
  1787. calls of
  1788. .CW execute
  1789. to avoid
  1790. problems with null matches.
  1791. For example, even given the leftmost-longest rule,
  1792. the expression
  1793. .CW a*
  1794. matches three times in the string
  1795. .CW ab
  1796. (the character
  1797. .CW a ,
  1798. the null string between the
  1799. .CW a
  1800. and
  1801. .CW b ,
  1802. and the final null string).
  1803. After returning a match for the
  1804. .CW a ,
  1805. .CW sam
  1806. must not match the null string before the
  1807. .CW b .
  1808. The algorithm starts
  1809. .CW execute
  1810. at the end of its previous match, and
  1811. if the match it returns
  1812. is null and abuts the previous match, rejects the match and advances
  1813. the initial position one character.
  1814. .SH 2
  1815. Memory allocation
  1816. .LP
  1817. The C language has no memory allocation primitives, although a standard
  1818. library routine,
  1819. .CW malloc ,
  1820. provides adequate service for simple programs.
  1821. For specific uses, however,
  1822. it can be better to write a custom allocator.
  1823. The allocator (or rather, pair of allocators) described here
  1824. work in both the terminal and host parts of
  1825. .CW sam .
  1826. They are designed for efficient manipulation of strings,
  1827. which are allocated and freed frequently and vary in length from essentially
  1828. zero to 32 Kbytes (very large strings are written to disc).
  1829. More important, strings may be large and change size often,
  1830. so to minimize memory usage it is helpful to reclaim and to coalesce the
  1831. unused portions of strings when they are truncated.
  1832. .PP
  1833. Objects to be allocated in
  1834. .CW sam
  1835. are of two flavors:
  1836. the first is C
  1837. .CW structs ,
  1838. which are small and often addressed by pointer variables;
  1839. the second is variable-sized arrays of characters
  1840. or integers whose
  1841. base pointer is always used to access them.
  1842. The memory allocator in
  1843. .CW sam
  1844. is therefore in two parts:
  1845. first, a traditional first-fit allocator that provides fixed storage for
  1846. .CW structs ;
  1847. and second, a garbage-compacting allocator that reduces storage
  1848. overhead for variable-sized objects, at the cost of some bookkeeping.
  1849. The two types of objects are allocated from adjoining arenas, with
  1850. the garbage-compacting allocator controlling the arena with higher addresses.
  1851. Separating into two arenas simplifies compaction and prevents fragmentation due
  1852. to immovable objects.
  1853. The access rules for garbage-compactable objects
  1854. (discussed in the next paragraph) allow them to be relocated, so when
  1855. the first-fit arena needs space, it moves the garbage-compacted arena
  1856. to higher addresses to make room. Storage is therefore created only
  1857. at successively higher addresses, either when more garbage-compacted
  1858. space is needed or when the first-fit arena pushes up the other arena.
  1859. .PP
  1860. Objects that may be compacted declare to the
  1861. allocator a cell that is guaranteed to be the sole repository of the
  1862. address of the object whenever a compaction can occur.
  1863. The compactor can then update the address when the object is moved.
  1864. For example, the implementation of type
  1865. .CW List
  1866. (really a variable-length array)
  1867. is:
  1868. .P1
  1869. typedef struct List{
  1870. int nused;
  1871. long *ptr;
  1872. }List;
  1873. .P2
  1874. The
  1875. .CW ptr
  1876. cell must always be used directly, and never copied. When a
  1877. .CW List
  1878. is to be created the
  1879. .CW List
  1880. structure is allocated in the ordinary first-fit arena
  1881. and its
  1882. .CW ptr
  1883. is allocated in the garbage-compacted arena.
  1884. A similar data type for strings, called
  1885. .CW String ,
  1886. stores variable-length character arrays of up to 32767 elements.
  1887. .PP
  1888. A related matter of programming style:
  1889. .CW sam
  1890. frequently passes structures by value, which
  1891. simplifies the code.
  1892. Traditionally, C programs have
  1893. passed structures by reference, but implicit allocation on
  1894. the stack is easier to use.
  1895. Structure passing is a relatively new feature of C
  1896. (it is not in the
  1897. standard reference manual for C\u\s-4\&14\s+4\d), and is poorly supported in most
  1898. commercial C compilers.
  1899. It's convenient and expressive, though,
  1900. and simplifies memory management by
  1901. avoiding the allocator altogether
  1902. and eliminating pointer aliases.
  1903. .SH 2
  1904. Data structures for manipulating files
  1905. .LP
  1906. Experience with
  1907. .CW jim
  1908. showed that the requirements
  1909. of the file data structure were few, but strict.
  1910. First, files need to be read and written quickly;
  1911. adding a fresh file must be painless.
  1912. Second, the implementation must place no arbitrary upper limit on
  1913. the number or sizes of files. (It should be practical to edit many files,
  1914. and files up to megabytes in length should be handled gracefully.)
  1915. This implies that files be stored on disc, not in main memory.
  1916. (Aficionados of virtual memory may argue otherwise, but the
  1917. implementation of virtual
  1918. memory in our system is not something to depend on
  1919. for good performance.)
  1920. Third, changes to files need be made by only two primitives:
  1921. deletion and insertion.
  1922. These are inverses of each other,
  1923. which simplifies the implementation of the undo operation.
  1924. Finally,
  1925. it must be easy and efficient to access the file, either
  1926. forwards or backwards, a byte at a time.
  1927. .PP
  1928. The
  1929. .CW File
  1930. data type is constructed from three simpler data structures that hold arrays
  1931. of characters.
  1932. Each of these types has an insertion and deletion operator, and the
  1933. insertion and deletion operators of the
  1934. .CW File
  1935. type itself are constructed from them.
  1936. .PP
  1937. The simplest type is the
  1938. .CW String ,
  1939. which is used to hold strings in main memory.
  1940. The code that manages
  1941. .CW Strings
  1942. guarantees that they will never be longer
  1943. than some moderate size, and in practice they are rarely larger than 8 Kbytes.
  1944. .CW Strings
  1945. have two purposes: they hold short strings like file names with little overhead,
  1946. and because they are deliberately small, they are efficient to modify.
  1947. They are therefore used as the data structure for in-memory caches.
  1948. .PP
  1949. The disc copy of the file is managed by a data structure called a
  1950. .CW Disc ,
  1951. which corresponds to a temporary file. A
  1952. .CW Disc
  1953. has no storage in main memory other than bookkeeping information;
  1954. the actual data being held is all on the disc.
  1955. To reduce the number of open files needed,
  1956. .CW sam
  1957. opens a dozen temporary UNIX files and multiplexes the
  1958. .CW Discs
  1959. upon them.
  1960. This permits many files to
  1961. be edited; the entire
  1962. .CW sam
  1963. source (48 files) may be edited comfortably with a single
  1964. instance of
  1965. .CW sam .
  1966. Allocating one temporary file per
  1967. .CW Disc
  1968. would strain the operating system's limit on the number of open files.
  1969. Also, spreading the traffic among temporary files keeps the files shorter,
  1970. and shorter files are more efficiently implemented by the UNIX
  1971. I/O subsystem.
  1972. .PP
  1973. A
  1974. .CW Disc
  1975. is an array of fixed-length blocks, each of which contains
  1976. between 1 and 4096 characters of active data.
  1977. (The block size of our UNIX file system is 4096 bytes.)
  1978. The block addresses within the temporary file and the length of each
  1979. block are stored in a
  1980. .CW List .
  1981. When changes are made the live part of blocks may change size.
  1982. Blocks are created and coalesced when necessary to try to keep the sizes
  1983. between 2048 and 4096 bytes.
  1984. An actively changing part of the
  1985. .CW Disc
  1986. therefore typically has about a kilobyte of slop that can be
  1987. inserted or deleted
  1988. without changing more than one block or affecting the block order.
  1989. When an insertion would overflow a block, the block is split, a new one
  1990. is allocated to receive the overflow, and the memory-resident list of blocks
  1991. is rearranged to reflect the insertion of the new block.
  1992. .PP
  1993. Obviously, going to the disc for every modification to the file is
  1994. prohibitively expensive.
  1995. The data type
  1996. .CW Buffer
  1997. consists of a
  1998. .CW Disc
  1999. to hold the data and a
  2000. .CW String
  2001. that acts as a cache.
  2002. This is the first of a series of caches throughout the data structures in
  2003. .CW sam.
  2004. The caches not only improve performance, they provide a way to organize
  2005. the flow of data, particularly in the communication between the host
  2006. and terminal.
  2007. This idea is developed below, in the section on communications.
  2008. .PP
  2009. To reduce disc traffic, changes to a
  2010. .CW Buffer
  2011. are mediated by a variable-length string, in memory, that acts as a cache.
  2012. When an insertion or deletion is made to a
  2013. .CW Buffer ,
  2014. if the change can be accommodated by the cache, it is done there.
  2015. If the cache becomes bigger than a block because of an insertion,
  2016. some of it is written to the
  2017. .CW Disc
  2018. and deleted from the cache.
  2019. If the change does not intersect the cache, the cache is flushed.
  2020. The cache is only loaded at the new position if the change is smaller than a block;
  2021. otherwise, it is sent directly to the
  2022. .CW Disc .
  2023. This is because
  2024. large changes are typically sequential,
  2025. whereupon the next change is unlikely to overlap the current one.
  2026. .PP
  2027. A
  2028. .CW File
  2029. comprises a
  2030. .CW String
  2031. to hold the file name and some ancillary data such as dot and the modified bit.
  2032. The most important components, though, are a pair of
  2033. .CW Buffers ,
  2034. one called the transcript and the other the contents.
  2035. Their use is described in the next section.
  2036. .PP
  2037. The overall structure is shown in Figure 5.
  2038. Although it may seem that the data is touched many times on its
  2039. way from the
  2040. .CW Disc ,
  2041. it is read (by one UNIX system call) directly into the cache of the
  2042. associated
  2043. .CW Buffer ;
  2044. no extra copy is done.
  2045. Similarly, when flushing the cache, the text is written
  2046. directly from the cache to disc.
  2047. Most operations act directly on the text in the cache.
  2048. A principle applied throughout
  2049. .CW sam
  2050. is that the fewer times the data is copied, the faster the program will run
  2051. (see also the paper by Waite\u\s-4\&15\s+4\d).
  2052. .KF
  2053. .PS
  2054. copy "fig5.pic"
  2055. .PE
  2056. .Cs
  2057. Figure 5. File data structures.
  2058. The temporary files are stored in the standard repository for such files
  2059. on the host system.
  2060. .Ce
  2061. .KE
  2062. .PP
  2063. The contents of a
  2064. .CW File
  2065. are accessed by a routine that
  2066. copies to a buffer a substring of a file starting at a specified offset.
  2067. To read a byte at a time, a
  2068. .CW File "" per-
  2069. array is loaded starting from a specified initial position,
  2070. and bytes may then be read from the array.
  2071. The implementation is done by a macro similar to the C standard I/O
  2072. .CW getc
  2073. macro.\u\s-4\&14\s+4\d
  2074. Because the reading may be done at any address, a minor change to the
  2075. macro allows the file to be read backwards.
  2076. This array is read-only; there is no
  2077. .CW putc .
  2078. .SH 2
  2079. Doing and undoing
  2080. .LP
  2081. .CW Sam
  2082. has an unusual method for managing changes to files.
  2083. The command language makes it easy to specify multiple variable-length changes
  2084. to a file millions of bytes long, and such changes
  2085. must be made efficiently if the editor is to be practical.
  2086. The usual techniques for inserting and deleting strings
  2087. are inadequate under these conditions.
  2088. The
  2089. .CW Buffer
  2090. and
  2091. .CW Disc
  2092. data structures are designed for efficient random access to long strings,
  2093. but care must be taken to avoid super-linear behavior when making
  2094. many changes simultaneously.
  2095. .PP
  2096. .CW Sam
  2097. uses a two-pass algorithm for making changes, and treats each file as a database
  2098. against which transactions are registered.
  2099. Changes are not made directly to the contents.
  2100. Instead, when a command is started, a `mark' containing
  2101. a sequence number is placed in the transcript
  2102. .CW Buffer ,
  2103. and each change made to the file, either an insertion or deletion
  2104. or a change to the file name,
  2105. is appended to the end of the transcript.
  2106. When the command is complete, the transcript is rewound to the
  2107. mark and applied to the contents.
  2108. .PP
  2109. One reason for separating evaluation from
  2110. application in this way is to simplify tracking the addresses of changes
  2111. made in the middle of a long sequence.
  2112. The two-pass algorithm also allows all changes to apply to the
  2113. .I original
  2114. data: no change can affect another change made in the same command.
  2115. This is particularly important when evaluating an
  2116. .CW x
  2117. command because it prevents regular expression matches
  2118. from stumbling over changes made earlier in the execution.
  2119. Also, the two-pass
  2120. algorithm is cleaner than the way other UNIX editors allow changes to
  2121. affect each other;
  2122. for example,
  2123. .CW ed 's
  2124. idioms to do things like delete every other line
  2125. depend critically on the implementation.
  2126. Instead,
  2127. .CW sam 's
  2128. simple model, in which all changes in a command occur effectively
  2129. simultaneously, is easy to explain and to understand.
  2130. .PP
  2131. The records in the transcript are of the form ``delete substring from
  2132. locations
  2133. 123 to 456'' and ``insert 11 characters `hello there' at location 789.''
  2134. (It is an error if the changes are not at monotonically greater
  2135. positions through the file.)
  2136. While the update is occurring, these numbers must be
  2137. offset by earlier changes, but that is straightforward and
  2138. local to the update routine;
  2139. moreover, all the numbers have been computed
  2140. before the first is examined.
  2141. .PP
  2142. Treating the file as a transaction system has another advantage:
  2143. undo is trivial.
  2144. All it takes is to invert the transcript after it has been
  2145. implemented, converting insertions
  2146. into deletions and vice versa, and saving them in a holding
  2147. .CW Buffer .
  2148. The `do' transcript can then be deleted from
  2149. the transcript
  2150. .CW Buffer
  2151. and replaced by the `undo' transcript.
  2152. If an undo is requested, the transcript is rewound and the undo transcript
  2153. executed.
  2154. Because the transcript
  2155. .CW Buffer
  2156. is not truncated after each command, it accumulates
  2157. successive changes.
  2158. A sequence of undo commands
  2159. can therefore back up the file arbitrarily,
  2160. which is more helpful than the more commonly implemented self-inverse form of undo.
  2161. .CW Sam "" (
  2162. provides no way to undo an undo, but if it were desired,
  2163. it would be easy to provide by re-interpreting the `do' transcript.)
  2164. Each mark in the transcript contains a sequence number and the offset into
  2165. the transcript of the previous mark, to aid in unwinding the transcript.
  2166. Marks also contain the value of dot and the modified bit so these can be
  2167. restored easily.
  2168. Undoing multiple files is easy; it merely demands undoing all files whose
  2169. latest change has the same sequence number as the current file.
  2170. .PP
  2171. Another benefit of having a transcript is that errors encountered in the middle
  2172. of a complicated command need not leave the files in an intermediate state.
  2173. By rewinding the transcript to the mark beginning the command,
  2174. the partial command can be trivially undone.
  2175. .PP
  2176. When the update algorithm was first implemented, it was unacceptably slow,
  2177. so a cache was added to coalesce nearby changes,
  2178. replacing multiple small changes by a single larger one.
  2179. This reduced the number
  2180. of insertions into the transaction
  2181. .CW Buffer ,
  2182. and made a dramatic improvement in performance,
  2183. but made it impossible
  2184. to handle changes in non-monotonic order in the file; the caching method
  2185. only works if changes don't overlap.
  2186. Before the cache was added, the transaction could in principle be sorted
  2187. if the changes were out of order, although
  2188. this was never done.
  2189. The current status is therefore acceptable performance with a minor
  2190. restriction on global changes, which is sometimes, but rarely, an annoyance.
  2191. .PP
  2192. The update algorithm obviously paws the data more than simpler
  2193. algorithms, but it is not prohibitively expensive;
  2194. the caches help.
  2195. (The principle of avoiding copying the data is still honored here,
  2196. although not as piously:
  2197. the data is moved from contents' cache to
  2198. the transcript's all at once and through only one internal buffer.)
  2199. Performance figures confirm the efficiency.
  2200. To read from a dead start a hundred kilobyte file on a VAX-11/750
  2201. takes 1.4 seconds of user time, 2.5 seconds of system time,
  2202. and 5 seconds of real time.
  2203. Reading the same file in
  2204. .CW ed
  2205. takes 6.0 seconds of user time, 1.7 seconds of system time,
  2206. and 8 seconds of real time.
  2207. .CW Sam
  2208. uses about half the CPU time.
  2209. A more interesting example is the one stated above:
  2210. inserting a character between every pair of characters in the file.
  2211. The
  2212. .CW sam
  2213. command is
  2214. .P1
  2215. ,y/@/ a/x/
  2216. .P2
  2217. and takes 3 CPU seconds per kilobyte of input file, of which
  2218. about a third is spent in the regular expression code.
  2219. This translates to about 500 changes per second.
  2220. .CW Ed
  2221. takes 1.5 seconds per kilobyte to make a similar change (ignoring newlines),
  2222. but cannot undo it.
  2223. The same example in
  2224. .CW ex ,\u\s-4\&9\s+4\d
  2225. a variant of
  2226. .CW ed
  2227. done at the University of California at Berkeley,
  2228. which allows one level of undoing, again takes 3 seconds.
  2229. In summary,
  2230. .CW sam 's
  2231. performance is comparable to that of other UNIX editors, although it solves
  2232. a harder problem.
  2233. .SH 2
  2234. Communications
  2235. .LP
  2236. The discussion so far has described the implementation of the host part of
  2237. .CW sam ;
  2238. the next few sections explain how a machine with mouse and bitmap display
  2239. can be engaged to improve interaction.
  2240. .CW Sam
  2241. is not the first editor to be written as two processes,\u\s-4\&16\s+4\d
  2242. but its implementation
  2243. has some unusual aspects.
  2244. .PP
  2245. There are several ways
  2246. .CW sam 's
  2247. host and terminal parts may be connected.
  2248. The first and simplest is to forgo the terminal part and use the host
  2249. part's command language to edit text on an ordinary terminal.
  2250. This mode is invoked by starting
  2251. .CW sam
  2252. with the
  2253. .CW -d
  2254. option.
  2255. With no options,
  2256. .CW sam
  2257. runs separate host and terminal programs,
  2258. communicating with a message protocol over the physical
  2259. connection that joins them.
  2260. Typically, the connection is an RS-232 link between a Blit
  2261. (the prototypical display for
  2262. .CW sam )
  2263. and a host running
  2264. the Ninth Edition of the UNIX operating system.\u\s-4\&8\s+4\d
  2265. (This is the version of the system used in the Computing Sciences Research
  2266. Center at AT&T Bell Laboratories [now Lucent Technologies, Bell Labs], where I work. Its relevant
  2267. aspects are discussed in the Blit paper.\u\s-4\&1\s+4\d)
  2268. The implementation of
  2269. .CW sam
  2270. for the SUN computer runs both processes on the same machine and
  2271. connects them by a pipe.
  2272. .PP
  2273. The low bandwidth of an RS-232 link
  2274. necessitated the split between
  2275. the two programs.
  2276. The division is a mixed blessing:
  2277. a program in two parts is much harder to write and to debug
  2278. than a self-contained one,
  2279. but the split makes several unusual configurations possible.
  2280. The terminal may be physically separated from the host, allowing the conveniences
  2281. of a mouse and bitmap display to be taken home while leaving the files at work.
  2282. It is also possible to run the host part on a remote machine:
  2283. .P1
  2284. sam -r host
  2285. .P2
  2286. connects to the terminal in the usual way, and then makes a call
  2287. across the network to establish the host part of
  2288. .CW sam
  2289. on the named machine.
  2290. Finally, it cross-connects the I/O to join the two parts.
  2291. This allows
  2292. .CW sam
  2293. to be run on machines that do not support bitmap displays;
  2294. for example,
  2295. .CW sam
  2296. is the editor of choice on our Cray X-MP/24.
  2297. .CW Sam
  2298. .CW -r
  2299. involves
  2300. .I three
  2301. machines: the remote host, the terminal, and the local host.
  2302. The local host's job is simple but vital: it passes the data
  2303. between the remote host and terminal.
  2304. .PP
  2305. The host and terminal exchange messages asynchronously
  2306. (rather than, say, as remote procedure calls) but there is no
  2307. error detection or correction
  2308. because, whatever the configuration, the connection is reliable.
  2309. Because the terminal handles mundane interaction tasks such as
  2310. popping up menus and interpreting the responses, the messages are about
  2311. data, not actions.
  2312. For example, the host knows nothing about what is displayed on the screen,
  2313. and when the user types a character, the message sent to the host says
  2314. ``insert a one-byte string at location 123 in file 7,'' not ``a character
  2315. was typed at the current position in the current file.''
  2316. In other words, the messages look very much like the transaction records
  2317. in the transcripts.
  2318. .PP
  2319. Either the host or terminal part of
  2320. .CW sam
  2321. may initiate a change to a file.
  2322. The command language operates on the host, while typing and some
  2323. mouse operations are executed directly in the terminal to optimize response.
  2324. Changes initiated by the host program must be transmitted to the terminal,
  2325. and
  2326. vice versa.
  2327. (A token is exchanged to determine which end is in control,
  2328. which means that characters typed while a time-consuming command runs
  2329. must be buffered and do not appear until the command is complete.)
  2330. To maintain consistent information,
  2331. the host and terminal track changes through a per-file
  2332. data structure that records what portions of the file
  2333. the terminal has received.
  2334. The data structure, called a
  2335. .CW Rasp
  2336. (a weak pun: it's a file with holes)
  2337. is held and updated by both the host and terminal.
  2338. A
  2339. .CW Rasp
  2340. is a list of
  2341. .CW Strings
  2342. holding those parts of the file known to the terminal,
  2343. separated by counts of the number of bytes in the interstices.
  2344. Of course, the host doesn't keep a separate copy of the data (it only needs
  2345. the lengths of the various pieces),
  2346. but the structure is the same on both ends.
  2347. .PP
  2348. The
  2349. .CW Rasp
  2350. in the terminal doubles as a cache.
  2351. Since the terminal keeps the text for portions of the file it has displayed,
  2352. it need not request data from the host when revisiting old parts of the file
  2353. or redrawing obscured windows, which speeds things up considerably
  2354. over low-speed links.
  2355. .PP
  2356. It's trivial for the terminal to maintain its
  2357. .CW Rasp ,
  2358. because all changes made on the terminal apply to parts of the file
  2359. already loaded there.
  2360. Changes made by the host are compared against the
  2361. .CW Rasp
  2362. during the update sequence after each command.
  2363. Small changes to pieces of the file loaded in the terminal
  2364. are sent in their entirety.
  2365. Larger changes, and changes that fall entirely in the holes,
  2366. are transmitted as messages without literal data:
  2367. only the lengths of the deleted and inserted strings are transmitted.
  2368. When a command is completed, the terminal examines its visible
  2369. windows to see if any holes in their
  2370. .CW Rasps
  2371. intersect the visible portion of the file.
  2372. It then requests the missing data from the host,
  2373. along with up to 512 bytes of surrounding data, to minimize
  2374. the number of messages when visiting a new portion of the file.
  2375. This technique provides a kind of two-level lazy evaluation for the terminal.
  2376. The first level sends a minimum of information about
  2377. parts of the file not being edited interactively;
  2378. the second level waits until a change is displayed before
  2379. transmitting the new data.
  2380. Of course,
  2381. performance is also helped by having the terminal respond immediately to typing
  2382. and simple mouse requests.
  2383. Except for small changes to active pieces of the file, which are
  2384. transmitted to the terminal without negotiation,
  2385. the terminal is wholly responsible for deciding what is displayed;
  2386. the host uses the
  2387. .CW Rasp
  2388. only to tell the terminal what might be relevant.
  2389. .PP
  2390. When a change is initiated by the host,
  2391. the messages to the terminal describing the change
  2392. are generated by the routine that applies the transcript of the changes
  2393. to the contents of the
  2394. .CW File .
  2395. Since changes are undone by the same update routine,
  2396. undoing requires
  2397. no extra code in the communications;
  2398. the usual messages describing changes to the file are sufficient
  2399. to back up the screen image.
  2400. .PP
  2401. The
  2402. .CW Rasp
  2403. is a particularly good example of the way caches are used in
  2404. .CW sam .
  2405. First, it facilitates access to the active portion of the text by placing
  2406. the busy text in main memory.
  2407. In so doing, it provides efficient access
  2408. to a large data structure that does not fit in memory.
  2409. Since the form of data is to be imposed by the user, not by the program,
  2410. and because characters will frequently be scanned sequentially,
  2411. files are stored as flat objects.
  2412. Caches help keep performance good and linear when working with such
  2413. data.
  2414. .PP
  2415. Second, the
  2416. .CW Rasp
  2417. and several of the other caches have some
  2418. .I read-ahead;
  2419. that is, the cache is loaded with more information than is needed for
  2420. the job immediately at hand.
  2421. When manipulating linear structures, the accesses are usually sequential,
  2422. and read-ahead can significantly reduce the average time to access the
  2423. next element of the object.
  2424. Sequential access is a common mode for people as well as programs;
  2425. consider scrolling through a document while looking for something.
  2426. .PP
  2427. Finally, like any good data structure,
  2428. the cache guides the algorithm, or at least the implementation.
  2429. The
  2430. .CW Rasp
  2431. was actually invented to control the communications between the host and
  2432. terminal parts, but I realized very early that it was also a form of
  2433. cache. Other caches were more explicitly intended to serve a double
  2434. purpose: for example, the caches in
  2435. .CW Files
  2436. that coalesce updates not only reduce traffic to the
  2437. transcript and contents
  2438. .CW Buffers ,
  2439. they also clump screen updates so that complicated changes to the
  2440. screen are achieved in
  2441. just a few messages to the terminal.
  2442. This saved me considerable work: I did not need to write special
  2443. code to optimize the message traffic to the
  2444. terminal.
  2445. Caches pay off in surprising ways.
  2446. Also, they tend to be independent, so their performance improvements
  2447. are multiplicative.
  2448. .SH 2
  2449. Data structures in the terminal
  2450. .LP
  2451. The terminal's job is to display and to maintain a consistent image of
  2452. pieces of the files being edited.
  2453. Because the text is always in memory, the data structures are
  2454. considerably simpler than those in the host part.
  2455. .PP
  2456. .CW Sam
  2457. typically has far more windows than does
  2458. .CW mux ,
  2459. the window system within which its Blit implementation runs.
  2460. .CW Mux
  2461. has a fairly small number of asynchronously updated windows;
  2462. .CW sam
  2463. needs a large number of synchronously updated windows that are
  2464. usually static and often fully obscured.
  2465. The different tradeoffs guided
  2466. .CW sam
  2467. away from the memory-intensive implementation of windows, called
  2468. .CW Layers ,\u\s-4\&17\s+4\d
  2469. used in
  2470. .CW mux.
  2471. Rather than depending on a complete bitmap image of the display for each window,
  2472. .CW sam
  2473. regenerates the image from its in-memory text
  2474. (stored in the
  2475. .CW Rasp )
  2476. when necessary, although it will use such an image if it is available.
  2477. Like
  2478. .CW Layers ,
  2479. though,
  2480. .CW sam
  2481. uses the screen bitmap as active storage in which to update the image using
  2482. .CW bitblt .\u\s-4\&18,19\s+4\d
  2483. The resulting organization, pictured in Figure 6,
  2484. has a global array of windows, called
  2485. .CW Flayers ,
  2486. each of which holds an image of a piece of text held in a data structure
  2487. called a
  2488. .CW Frame ,
  2489. which in turn represents
  2490. a rectangular window full of text displayed in some
  2491. .CW Bitmap .
  2492. Each
  2493. .CW Flayer
  2494. appears in a global list that orders them all front-to-back
  2495. on the display, and simultaneously as an element of a per-file array
  2496. that holds all the open windows for that file.
  2497. The complement in the terminal of the
  2498. .CW File
  2499. on the host is called a
  2500. .CW Text ;
  2501. each connects its
  2502. .CW Flayers
  2503. to the associated
  2504. .CW Rasp .
  2505. .KF
  2506. .PS
  2507. copy "fig6.pic"
  2508. .PE
  2509. .Cs
  2510. Figure 6. Data structures in the terminal.
  2511. .CW Flayers
  2512. are also linked together into a front-to-back list.
  2513. .CW Boxes
  2514. are discussed in the next section.
  2515. .Ce
  2516. .KE
  2517. .PP
  2518. The
  2519. .CW Bitmap
  2520. for a
  2521. .CW Frame
  2522. contains the image of the text.
  2523. For a fully visible window, the
  2524. .CW Bitmap
  2525. will be the screen (or at least the
  2526. .CW Layer
  2527. in which
  2528. .CW sam
  2529. is being run),
  2530. while for partially obscured windows the
  2531. .CW Bitmap
  2532. will be off-screen.
  2533. If the window is fully obscured, the
  2534. .CW Bitmap
  2535. will be null.
  2536. .PP
  2537. The
  2538. .CW Bitmap
  2539. is a kind of cache.
  2540. When making changes to the display, most of the original image will
  2541. look the same in the final image, and the update algorithms exploit this.
  2542. The
  2543. .CW Frame
  2544. software updates the image in the
  2545. .CW Bitmap
  2546. incrementally; the
  2547. .CW Bitmap
  2548. is not just an image, it is a data structure.\u\s-4\&18,19\s+4\d
  2549. The job of the software that updates the display is therefore
  2550. to use as much as possible of the existing image (converting the
  2551. text from ASCII characters to pixels is expensive) in a sort of two-dimensional
  2552. string insertion algorithm.
  2553. The details of this process are described in the next section.
  2554. .PP
  2555. The
  2556. .CW Frame
  2557. software has no code to support overlapping windows;
  2558. its job is to keep a single
  2559. .CW Bitmap
  2560. up to date.
  2561. It falls to the
  2562. .CW Flayer
  2563. software to multiplex the various
  2564. .CW Bitmaps
  2565. onto the screen.
  2566. The problem of maintaining overlapping
  2567. .CW Flayers
  2568. is easier than for
  2569. .CW Layers \u\s-4\&17\s+4\d
  2570. because changes are made synchronously and because the contents of the window
  2571. can be reconstructed from the data stored in the
  2572. .CW Frame ;
  2573. the
  2574. .CW Layers
  2575. software
  2576. makes no such assumptions.
  2577. In
  2578. .CW sam ,
  2579. the window being changed is almost always fully visible, because the current
  2580. window is always fully visible, by construction.
  2581. However, when multi-file changes are being made, or when
  2582. more than one window is open on a file,
  2583. it may be necessary to update partially obscured windows.
  2584. .PP
  2585. There are three cases: the window is
  2586. fully visible, invisible (fully obscured), or partially visible.
  2587. If fully visible, the
  2588. .CW Bitmap
  2589. is part of the screen, so when the
  2590. .CW Flayer
  2591. update routine calls the
  2592. .CW Frame
  2593. update routine, the screen will be updated directly.
  2594. If the window is invisible,
  2595. there is no associated
  2596. .CW Bitmap ,
  2597. and all that is necessary is to update the
  2598. .CW Frame
  2599. data structure, not the image.
  2600. If the window is partially visible, the
  2601. .CW Frame
  2602. routine is called to update the image in the off-screen
  2603. .CW Bitmap ,
  2604. which may require regenerating it from the text of the window.
  2605. The
  2606. .CW Flayer
  2607. code then clips this
  2608. .CW Bitmap
  2609. against the
  2610. .CW Bitmaps
  2611. of all
  2612. .CW Frames
  2613. in front of the
  2614. .CW Frame
  2615. being modified, and the remainder is copied to the display.
  2616. .PP
  2617. This is much faster than recreating the image off-screen
  2618. for every change, or clipping all the changes made to the image
  2619. during its update.
  2620. Unfortunately, these caches can also consume prohibitive amounts of
  2621. memory, so they are freed fairly liberally \(em after every change to the
  2622. front-to-back order of the
  2623. .CW Flayers .
  2624. The result is that
  2625. the off-screen
  2626. .CW Bitmaps
  2627. exist only while multi-window changes are occurring,
  2628. which is the only time the performance improvement they provide is needed.
  2629. Also, the user interface causes fully-obscured windows to be the
  2630. easiest to make \(em
  2631. creating a canonically sized and placed window requires only a button click
  2632. \(em which reduces the need for caching still further.
  2633. .PP
  2634. .SH 2
  2635. Screen update
  2636. .LP
  2637. Only two low-level primitives are needed for incremental update:
  2638. .CW bitblt ,
  2639. which copies rectangles of pixels, and
  2640. .CW string
  2641. (which in turn calls
  2642. .CW bitblt ),
  2643. which draws a null-terminated character string in a
  2644. .CW Bitmap .
  2645. A
  2646. .CW Frame
  2647. contains a list of
  2648. .CW Boxes ,
  2649. each of which defines a horizontal strip of text in the window
  2650. (see Figure 7).
  2651. A
  2652. .CW Box
  2653. has a character string
  2654. .CW str ,
  2655. and a
  2656. .CW Rectangle
  2657. .CW rect
  2658. that defines the location of the strip in the window.
  2659. (The text in
  2660. .CW str
  2661. is stored in the
  2662. .CW Box
  2663. separately from the
  2664. .CW Rasp
  2665. associated with the window's file, so
  2666. .CW Boxes
  2667. are self-contained.)
  2668. The invariant is that
  2669. the image of the
  2670. .CW Box
  2671. can be reproduced by calling
  2672. .CW string
  2673. with argument
  2674. .CW str
  2675. to draw the string in
  2676. .CW rect ,
  2677. and the resulting picture fits perfectly within
  2678. .CW rect .
  2679. In other words, the
  2680. .CW Boxes
  2681. define the tiling of the window.
  2682. The tiling may be complicated by long lines of text, which
  2683. are folded onto the next line.
  2684. Some editors use horizontal scrolling to avoid this complication,
  2685. but to be comfortable this technique requires that lines not be
  2686. .I too
  2687. long;
  2688. .CW sam
  2689. has no such restriction.
  2690. Also, and perhaps more importantly, UNIX programs and terminals traditionally fold
  2691. long lines to make their contents fully visible.
  2692. .PP
  2693. Two special kinds of
  2694. .CW Boxes
  2695. contain a single
  2696. character: either a newline or a tab.
  2697. Newlines and tabs are white space.
  2698. A newline
  2699. .CW Box
  2700. always extends to the right edge of the window,
  2701. forcing the following
  2702. .CW Box
  2703. to the next line.
  2704. The width of a tab depends on where it is located:
  2705. it forces the next
  2706. .CW Box
  2707. to begin at a tab location.
  2708. Tabs also
  2709. have a minimum width equivalent to a blank (blanks are
  2710. drawn by
  2711. .CW string
  2712. and are not treated specially); newlines have a minimum width of zero.
  2713. .KF
  2714. .PS
  2715. copy "fig7.pic"
  2716. .PE
  2717. .sp .5
  2718. .Cs
  2719. Figure 7. A line of text showing its
  2720. .CW Boxes .
  2721. The first two blank
  2722. .CW Boxes
  2723. contain tabs; the last contains a newline.
  2724. Spaces are handled as ordinary characters.
  2725. .Ce
  2726. .KE
  2727. .PP
  2728. The update algorithms always use the
  2729. .CW Bitmap
  2730. image of the text (either the display or cache
  2731. .CW Bitmap );
  2732. they never examine the characters within a
  2733. .CW Box
  2734. except when the
  2735. .CW Box
  2736. needs to be split in two.
  2737. Before a change, the window consists of a tiling of
  2738. .CW Boxes ;
  2739. after the change the window is tiled differently.
  2740. The update algorithms rearrange the tiles in place, without
  2741. backup storage.
  2742. The algorithms are not strictly optimal \(em for example, they can
  2743. clear a pixel that is later going to be written upon \(em
  2744. but they never move a tile that doesn't need to be moved,
  2745. and they move each tile at most once.
  2746. .CW Frinsert
  2747. on a Blit can absorb over a thousand characters a second if the strings
  2748. being inserted are a few tens of characters long.
  2749. .PP
  2750. Consider
  2751. .CW frdelete .
  2752. Its job is to delete a substring from a
  2753. .CW Frame
  2754. and restore the image of the
  2755. .CW Frame .
  2756. The image of a substring has a peculiar shape (see Figure 2) comprising
  2757. possibly a partial line,
  2758. zero or more full lines,
  2759. and possibly a final partial line.
  2760. For reference, call this the
  2761. .I
  2762. Z-shape.
  2763. .R
  2764. .CW Frdelete
  2765. begins by splitting, if necessary, the
  2766. .CW Boxes
  2767. containing the ends of
  2768. the substring so the substring begins and ends on
  2769. .CW Box
  2770. boundaries.
  2771. Because the substring is being deleted, its image is not needed,
  2772. so the Z-shape is then cleared.
  2773. Then, tiles (that is, the images of
  2774. .CW Boxes )
  2775. are copied, using
  2776. .CW bitblt ,
  2777. from immediately after the Z-shape to
  2778. the beginning of the Z-shape,
  2779. resulting in a new Z-shape.
  2780. .CW Boxes "" (
  2781. whose contents would span two lines in the new position must first be split.)
  2782. .PP
  2783. Copying the remainder of the
  2784. .CW Frame
  2785. tile by tile
  2786. this way will clearly accomplish the deletion but eventually,
  2787. typically when the copying algorithm encounters a tab or newline,
  2788. the old and new
  2789. .CW x
  2790. coordinates of the tile
  2791. to be copied are the same.
  2792. This correspondence implies
  2793. that the Z-shape has its beginning and ending edges aligned
  2794. vertically, and a sequence of at most two
  2795. .CW bitblts
  2796. can be used to copy the remaining tiles.
  2797. The last step is to clear out the resulting empty space at the bottom
  2798. of the window;
  2799. the number of lines to be cleared is the number of complete lines in the
  2800. Z-shape closed by the final
  2801. .CW bitblts.
  2802. The final step is to merge horizontally adjacent
  2803. .CW Boxes
  2804. of plain text.
  2805. The complete source to
  2806. .CW frdelete
  2807. is less than 100 lines of C.
  2808. .PP
  2809. .CW frinsert
  2810. is more complicated because it must do four passes:
  2811. one to construct the
  2812. .CW Box
  2813. list for the inserted string,
  2814. one to reconnoitre,
  2815. one to copy (in opposite order to
  2816. .CW frdelete )
  2817. the
  2818. .CW Boxes
  2819. to make the hole for the new text,
  2820. and finally one to copy the new text into place.
  2821. Overall, though,
  2822. .CW frinsert
  2823. has a similar flavor to
  2824. .CW frdelete ,
  2825. and needn't be described further.
  2826. .CW Frinsert
  2827. and its subsidiary routines comprise 211 lines of C.
  2828. .PP
  2829. The terminal source code is 3024 lines of C,
  2830. and the host source is 5797 lines.
  2831. .SH
  2832. Discussion
  2833. .SH 2
  2834. History
  2835. .LP
  2836. The immediate ancestor of
  2837. .CW sam
  2838. was the original text editor for the Blit, called
  2839. .CW jim .
  2840. .CW Sam
  2841. inherited
  2842. .CW jim 's
  2843. two-process structure and mouse language almost unchanged, but
  2844. .CW jim
  2845. suffered from several drawbacks that were addressed in the design of
  2846. .CW sam .
  2847. The most important of these was the lack of a command language.
  2848. Although
  2849. .CW jim
  2850. was easy to use for simple editing, it provided no direct help with
  2851. large or repetitive editing tasks. Instead, it provided a command to pass
  2852. selected text through a shell pipeline,
  2853. but this was no more satisfactory than could be expected of a stopgap measure.
  2854. .PP
  2855. .CW Jim
  2856. was written primarily as a vehicle for experimenting with a mouse-based
  2857. interface to text, and the experiment was successful.
  2858. .CW Jim
  2859. had some spin-offs:
  2860. .CW mux ,
  2861. the second window system for the Blit, is essentially a multiplexed
  2862. version of the terminal part of
  2863. .CW jim ;
  2864. and the debugger
  2865. .CW pi 's
  2866. user interface\u\s-4\&20\s+4\d was closely modeled on
  2867. .CW jim 's.
  2868. But after a couple of years,
  2869. .CW jim
  2870. had become difficult to maintain and limiting to use,
  2871. and its replacement was overdue.
  2872. .PP
  2873. I began the design of
  2874. .CW sam
  2875. by asking
  2876. .CW jim
  2877. customers what they wanted.
  2878. This was probably a mistake; the answers were essentially a list of features
  2879. to be found in other editors, which did not provide any of the
  2880. guiding principles I was seeking.
  2881. For instance, one common request was for a ``global substitute,''
  2882. but no one suggested how to provide it within a cut-and-paste editor.
  2883. I was looking for a scheme that would
  2884. support such specialized features comfortably in the context of some
  2885. general command language.
  2886. Ideas were not forthcoming, though, particularly given my insistence
  2887. on removing all limits on file sizes, line lengths and so on.
  2888. Even worse, I recognized that, since the mouse could easily
  2889. indicate a region of the screen that was not an integral number of lines,
  2890. the command language would best forget about newlines altogether,
  2891. and that meant the command language had to treat the file as a single
  2892. string, not an array of lines.
  2893. .PP
  2894. Eventually, I decided that thinking was not getting me very far and it was
  2895. time to try building.
  2896. I knew that the terminal part could be built easily \(em
  2897. that part of
  2898. .CW jim
  2899. behaved acceptably well \(em and that most of the hard work was going
  2900. to be in the host part: the file interface, command interpreter and so on.
  2901. Moreover, I had some ideas about how the architecture of
  2902. .CW jim
  2903. could be improved without destroying its basic structure, which I liked
  2904. in principle but which hadn't worked out as well as I had hoped.
  2905. So I began by designing the file data structure,
  2906. starting with the way
  2907. .CW jim
  2908. worked \(em comparable to a single structure merging
  2909. .CW Disc
  2910. and
  2911. .CW Buffer ,
  2912. which I split to make the cache more general
  2913. \(em and thinking about how global substitute could be implemented.
  2914. The answer was clearly that it had to be done in two passes,
  2915. and the transcript-oriented implementation fell out naturally.
  2916. .PP
  2917. .CW Sam
  2918. was written bottom-up,
  2919. starting from the data structures and algorithms for manipulating text,
  2920. through the command language and up to the code for maintaining
  2921. the display.
  2922. In retrospect, it turned out well, but this implementation method is
  2923. not recommended in general.
  2924. There were several times when I had a large body of interesting code
  2925. assembled and no clue how to proceed with it.
  2926. The command language, in particular, took almost a year to figure out,
  2927. but can be implemented (given what was there at the beginning of that year)
  2928. in a day or two. Similarly, inventing the
  2929. .CW Rasp
  2930. data structure delayed the
  2931. connection of the host and terminal pieces by another few months.
  2932. .CW Sam
  2933. took about two years to write, although only about four months were
  2934. spent actually working on it.
  2935. .PP
  2936. Part of the design process was unusual:
  2937. the subset of the protocol that maintains the
  2938. .CW Rasp
  2939. was simulated, debugged
  2940. and verified by an automatic protocol analyzer,\u\s-4\&21\s+4\d and was bug-free
  2941. from the start.
  2942. The rest of the protocol, concerned mostly
  2943. with keeping menus up to date,
  2944. was unfortunately too unwieldy for such analysis,
  2945. and was debugged by more traditional methods, primarily
  2946. by logging in a file all messages in and out of the host.
  2947. .SH 2
  2948. Reflections
  2949. .LP
  2950. .CW Sam
  2951. is essentially the only interactive editor used by the sixty or so members of
  2952. the computing science research center in which I work.
  2953. The same could not be said of
  2954. .CW jim ;
  2955. the lack of a command language kept some people from adopting it.
  2956. The union of a user interface as comfortable as
  2957. .CW jim 's
  2958. with a command language as powerful as
  2959. .CW ed 's†
  2960. .FS
  2961. .vs 9
  2962. †The people who criticize
  2963. .CW ed
  2964. as an interactive program often forget that it and its close relative
  2965. .CW sed \u\s-4\&7\s+4\d
  2966. still thrive as programmable editors. The strength of these programs is
  2967. independent of their convenience for interactive editing.
  2968. .br
  2969. .vs
  2970. .FE
  2971. is essential to
  2972. .CW sam 's
  2973. success.
  2974. When
  2975. .CW sam
  2976. was first made available to the
  2977. .CW jim
  2978. community,
  2979. almost everyone switched to it within two or three days.
  2980. In the months that followed, even people who had never adopted
  2981. .CW jim
  2982. started using
  2983. .CW sam
  2984. exclusively.
  2985. .PP
  2986. To be honest,
  2987. .CW ed
  2988. still gets occasional use, but usually when
  2989. something quick needs to be done and the overhead of
  2990. downloading the terminal part of
  2991. .CW sam
  2992. isn't worth the trouble.
  2993. Also, as a `line' editor,
  2994. .CW sam
  2995. .CW -d
  2996. is a bit odd;
  2997. when using a good old ASCII terminal, it's comforting to have
  2998. a true line editor.
  2999. But it is fair to say that
  3000. .CW sam 's
  3001. command language has displaced
  3002. .CW ed 's
  3003. for most of the complicated editing that has kept line editors
  3004. (that is, command-driven editors) with us.
  3005. .PP
  3006. .CW Sam 's
  3007. command language is even fancier than
  3008. .CW ed 's,
  3009. and most
  3010. .CW sam
  3011. customers don't come near to using all its capabilities.
  3012. Does it need to be so sophisticated?
  3013. I think the answer is yes, for two reasons.
  3014. .PP
  3015. First, the
  3016. .I model
  3017. for
  3018. .CW sam 's
  3019. command language is really relatively simple, and certainly simpler than that of
  3020. .CW ed .
  3021. For instance, there is only one kind of textual loop in
  3022. .CW sam
  3023. \(em the
  3024. .CW x
  3025. command \(em
  3026. while
  3027. .CW ed
  3028. has three (the
  3029. .CW g
  3030. command, the global flag on substitutions, and the implicit loop over
  3031. lines in multi-line substitutions).
  3032. Also,
  3033. .CW ed 's
  3034. substitute command is necessary to make changes within lines, but in
  3035. .CW sam
  3036. the
  3037. .CW s
  3038. command is more of a familiar convenience than a necessity;
  3039. .CW c
  3040. and
  3041. .CW t
  3042. can do all the work.
  3043. .PP
  3044. Second,
  3045. given a community that expects an editor to be about as powerful as
  3046. .CW ed ,
  3047. it's hard to see how
  3048. .CW sam
  3049. could really be much simpler and still satisfy that expectation.
  3050. People want to do ``global substitutes,'' and most are content
  3051. to have the recipe for that and a few other fancy changes.
  3052. The sophistication of the command language is really just a veneer
  3053. over a design that makes it possible to do global substitutes
  3054. in a screen editor.
  3055. Some people will always want something more, however, and it's gratifying to
  3056. be able to provide it.
  3057. The real power of
  3058. .CW sam 's
  3059. command language comes from composability of the operators, which is by
  3060. nature orthogonal to the underlying model.
  3061. In other words,
  3062. .CW sam
  3063. is not itself complex, but it makes complex things possible.
  3064. If you don't want to do anything complex, you can ignore the
  3065. complexity altogether, and many people do so.
  3066. .PP
  3067. Sometimes I am asked the opposite question: why didn't I just make
  3068. .CW sam
  3069. a real programmable editor, with macros and variables and so on?
  3070. The main reason is a matter of taste: I like the editor
  3071. to be the same every time I use it.
  3072. There is one technical reason, though:
  3073. programmability in editors is largely a workaround for insufficient
  3074. interactivity.
  3075. Programmable editors are used to make particular, usually short-term,
  3076. things easy to do, such as by providing shorthands for common actions.
  3077. If things are generally easy to do in the first place,
  3078. shorthands are not as helpful.
  3079. .CW Sam
  3080. makes common editing operations very easy, and the solutions to
  3081. complex editing problems seem commensurate with the problems themselves.
  3082. Also, the ability to edit the
  3083. .CW sam
  3084. window makes it easy to repeat commands \(em it only takes a mouse button click
  3085. to execute a command again.
  3086. .SH 2
  3087. Pros and cons
  3088. .LP
  3089. .CW Sam
  3090. has several other good points,
  3091. and its share of problems.
  3092. Among the good things is the idea of
  3093. structural regular expressions,
  3094. whose usefulness has only begun to be explored.
  3095. They were arrived at serendipitously when I attempted to distill the essence of
  3096. .CW ed 's
  3097. way of doing global substitution and recognized that the looping command in
  3098. .CW ed
  3099. was implicitly imposing a structure (an array of lines) on the file.
  3100. .PP
  3101. Another of
  3102. .CW sam 's
  3103. good things is its undo capability.
  3104. I had never before used an editor with a true undo,
  3105. but I would never go back now.
  3106. Undo
  3107. .I must
  3108. be done well, but if it is, it can be relied on.
  3109. For example,
  3110. it's safe to experiment if you're not sure how to write some intricate command,
  3111. because if you make a mistake, it can be fixed simply and reliably.
  3112. I learned two things about undo from writing
  3113. .CW sam :
  3114. first, it's easy to provide if you design it in from the beginning, and
  3115. second, it's necessary, particularly if the system has some subtle
  3116. properties that may be unfamiliar or error-prone for users.
  3117. .PP
  3118. .CW Sam 's
  3119. lack of internal limits and sizes is a virtue.
  3120. Because it avoids all fixed-size tables and data structures,
  3121. .CW sam
  3122. is able to make global changes to files that some of our other
  3123. tools cannot even read.
  3124. Moreover, the design keeps the performance linear when doing such
  3125. operations, although I must admit
  3126. .CW sam
  3127. does get slow when editing a huge file.
  3128. .PP
  3129. Now, the problems.
  3130. Externally, the most obvious is that it is poorly integrated into the
  3131. surrounding window system.
  3132. By design, the user interface in
  3133. .CW sam
  3134. feels almost identical to that of
  3135. .CW mux ,
  3136. but a thick wall separates text in
  3137. .CW sam
  3138. from the programs running in
  3139. .CW mux .
  3140. For instance, the `snarf buffer' in
  3141. .CW sam
  3142. must be maintained separately from that in
  3143. .CW mux .
  3144. This is regrettable, but probably necessary given the unusual configuration
  3145. of the system, with a programmable terminal on the far end of an RS-232 link.
  3146. .PP
  3147. .CW Sam
  3148. is reliable; otherwise, people wouldn't use it.
  3149. But it was written over such a long time, and has so many new (to me)
  3150. ideas in it, that I would like to see it done over again to clean
  3151. up the code and remove many of the lingering problems in the implementation.
  3152. The worst part is in the interconnection of the host and terminal parts,
  3153. which might even be able to go away in a redesign for a more
  3154. conventional window system.
  3155. The program must be split in two to use the terminal effectively,
  3156. but the low bandwidth of the connection forces the separation to
  3157. occur in an inconvenient part of the design if performance is to be acceptable.
  3158. A simple remote procedure call
  3159. protocol driven by the host, emitting only graphics
  3160. commands, would be easy to write but wouldn't have nearly the
  3161. necessary responsiveness. On the other hand, if the terminal were in control
  3162. and requested much simpler file services from the host, regular expression
  3163. searches would require that the terminal read the entire file over its RS-232
  3164. link, which would be unreasonably slow.
  3165. A compromise in which either end can take control is necessary.
  3166. In retrospect, the communications protocol should have been
  3167. designed and verified formally, although I do not know of any tool
  3168. that can adequately relate the protocol to
  3169. its implementation.
  3170. .PP
  3171. Not all of
  3172. .CW sam 's
  3173. users are comfortable with its command language, and few are adept.
  3174. Some (venerable) people use a sort of
  3175. .CW ed \& ``
  3176. subset'' of
  3177. .CW sam 's
  3178. command language,
  3179. and even ask why
  3180. .CW sam 's
  3181. command language is not exactly
  3182. .CW ed 's.
  3183. (The reason, of course, is that
  3184. .CW sam 's
  3185. model for text does not include newlines, which are central to
  3186. .CW ed .
  3187. Making the text an array of newlines to the command language would
  3188. be too much of a break from the seamless model provided by the mouse.
  3189. Some editors, such as
  3190. .CW vi ,
  3191. are willing to make this break, though.)
  3192. The difficulty is that
  3193. .CW sam 's
  3194. syntax is so close to
  3195. .CW ed 's
  3196. that people believe it
  3197. .I should
  3198. be the same.
  3199. I thought, with some justification in hindsight,
  3200. that making
  3201. .CW sam
  3202. similar to
  3203. .CW ed
  3204. would make it easier to learn and to accept.
  3205. But I may have overstepped and raised the users'
  3206. expectations too much.
  3207. It's hard to decide which way to resolve this problem.
  3208. .PP
  3209. Finally, there is a tradeoff in
  3210. .CW sam
  3211. that was decided by the environment in which it runs:
  3212. .CW sam
  3213. is a multi-file editor, although in a different system there might instead be
  3214. multiple single-file editors.
  3215. The decision was made primarily because starting a new program in a Blit is
  3216. time-consuming.
  3217. If the choice could be made freely, however, I would
  3218. still choose the multi-file architecture, because it allows
  3219. groups of files to be handled as a unit;
  3220. the usefulness of the multi-file commands is incontrovertible.
  3221. It is delightful to have the source to an entire program
  3222. available at your fingertips.
  3223. .SH
  3224. Acknowledgements
  3225. .LP
  3226. Tom Cargill suggested the idea behind the
  3227. .CW Rasp
  3228. data structure.
  3229. Norman Wilson and Ken Thompson influenced the command language.
  3230. This paper was improved by comments from
  3231. Al Aho,
  3232. Jon Bentley,
  3233. Chris Fraser,
  3234. Gerard Holzmann,
  3235. Brian Kernighan,
  3236. Ted Kowalski,
  3237. Doug McIlroy
  3238. and
  3239. Dennis Ritchie.