replica 6.2 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316
  1. .TH REPLICA 8
  2. .SH NAME
  3. applychanges, applylog, compactdb, updatedb \- simple client-server replica management
  4. .SH SYNOPSIS
  5. .B replica/compactdb
  6. .I db
  7. .br
  8. .B replica/updatedb
  9. [
  10. .B -cl
  11. ]
  12. [
  13. .B -p
  14. .I proto
  15. ]
  16. [
  17. .B -r
  18. .I root
  19. ]
  20. [
  21. .B -t
  22. .I now
  23. .I n
  24. ]
  25. [
  26. .B -u
  27. .I uid
  28. ]
  29. [
  30. .B -x
  31. .I path
  32. ] ...
  33. .I db
  34. .br
  35. .B replica/applylog
  36. [
  37. .B -nuv
  38. ]
  39. [
  40. .B -c
  41. .I name
  42. ]...
  43. [
  44. .B -s
  45. .I name
  46. ]...
  47. .I clientdb
  48. .I clientroot
  49. .I serverroot
  50. [
  51. .I path ...
  52. ]
  53. .br
  54. .B replica/applychanges
  55. [
  56. .B -nuv
  57. ]
  58. [
  59. .B -p
  60. .I proto
  61. ]
  62. [
  63. .B -x
  64. .I path
  65. ] ...
  66. .I clientdb
  67. .I clientroot
  68. .I serverroot
  69. [
  70. .I path ...
  71. ]
  72. .SH DESCRIPTION
  73. These four tools collectively provide simple log-based
  74. client-server replica management.
  75. The shell scripts described in
  76. .IR replica (1)
  77. provide a more polished interface.
  78. .PP
  79. Both client and server maintain textual databases of file system
  80. metadata. Each line is of the form
  81. .ift .sp 0.5
  82. .ifn .sp
  83. \h'0.25i'
  84. .I path
  85. .I mode
  86. .I uid
  87. .I gid
  88. .I mtime
  89. .I length
  90. .ift .sp 0.5
  91. .ifn .sp
  92. Later entries for a
  93. .I path
  94. supersede previous ones.
  95. A line with the string
  96. .B REMOVED
  97. in the
  98. .I mode
  99. field annuls all previous entries for that
  100. .IR path .
  101. The entries in a file are typically kept sorted by
  102. .I path
  103. but need not be.
  104. These properties facilitate updating the database atomically
  105. by appending to it.
  106. .I Compactdb
  107. reads in a database and writes out an equivalent one,
  108. sorted by path and without outdated or annulled records.
  109. .PP
  110. A replica is further described on the server by a textual
  111. log listing creation and deletion of files and changes
  112. to file contents and metadata.
  113. Each line is of the form:
  114. .ift .sp 0.5
  115. .ifn .sp
  116. \h'0.25i'
  117. .I time
  118. .I gen
  119. .I verb
  120. .I path
  121. .I serverpath
  122. .I mode
  123. .I uid
  124. .I gid
  125. .I mtime
  126. .I length
  127. .ift .sp 0.5
  128. .ifn .sp
  129. The
  130. .I time
  131. and
  132. .I gen
  133. fields are both decimal numbers, providing an ordering
  134. for log entries so that incremental tools need not process
  135. the whole log each time they are run.
  136. The
  137. .IR verb ,
  138. a single character,
  139. describes the event:
  140. addition of a file
  141. .RB ( a ),
  142. deletion of a file
  143. .RB ( d ),
  144. a change to a file's contents
  145. .RB ( c ),
  146. or a change to a file's metadata
  147. .RB ( m ).
  148. .I Path
  149. is the file path on the client;
  150. .I serverpath
  151. the path on the server (these are different when the
  152. optional fifth field in a proto file line is given;
  153. see
  154. .IR proto (2)).
  155. .IR Mode ,
  156. .IR uid ,
  157. .IR gid ,
  158. and
  159. .I mtime
  160. are the files metadata as in the
  161. .B Dir
  162. structure (see
  163. .IR stat (5)).
  164. For deletion events, the metadata is that of the deleted file.
  165. For other events, the metadata is that after the event.
  166. .PP
  167. .I Updatedb
  168. scans the file system rooted at
  169. .I root
  170. for changes not present in
  171. .IR db ,
  172. noting them by appending new entries to the database
  173. and by writing log events to standard output.
  174. The
  175. .B -c
  176. option causes
  177. .I updatedb
  178. to consider only file and metadata changes, ignoring file additions and deletions.
  179. By default, the log events have
  180. .I time
  181. set to the current system time
  182. and use incrementing
  183. .I gen
  184. numbers starting at 0.
  185. The
  186. .B -t
  187. option can be used to specify a different time and starting number.
  188. If the
  189. .B -u
  190. option is given, all database entries and log events will use
  191. .I uid
  192. rather than the actual uids.
  193. The
  194. .B -x
  195. option (which may be specified multiple times) excludes the named path
  196. and all its children from the scan.
  197. If the
  198. .B -l
  199. option is given, the database is not changed and the
  200. .I time
  201. and
  202. .I gen
  203. fields are omitted from the log events;
  204. the resulting output is intended to be a human-readable
  205. summary of file system activity since the last scan.
  206. .PP
  207. .I Applylog
  208. is used to propagate changes from server to client.
  209. It applies the changes listed in a log
  210. (read from standard input)
  211. to the file system rooted at
  212. .IR clientroot ,
  213. copying files when necessary from the file system rooted at
  214. .IR serverroot .
  215. By default,
  216. .I applylog
  217. does not attempt to set the uid on files; the
  218. .B -u
  219. flag enables this.
  220. .I Applylog
  221. will not overwrite local changes made to replicated files.
  222. When it detects such conflicts, by default it prints an error describing
  223. the conflict and takes no action.
  224. If the
  225. .B -c
  226. flag is given,
  227. .I applylog
  228. still takes no action for files beginning with the given names,
  229. but does so silently and will not
  230. report the conflicts in the future.
  231. (The conflict is resolved in favor of the client.)
  232. The
  233. .B -s
  234. is similar but causes
  235. .I applylog
  236. to overwrite the local changes.
  237. (The conflict is resolved in favor of the server.)
  238. .PP
  239. .I Applychanges
  240. is, in some sense, the opposite of
  241. .IR applylog ;
  242. it scans the client file system for changes, and applies
  243. those changes to the server file system.
  244. .I Applychanges
  245. will not overwrite remote changes made to replicated files.
  246. For example, if a file is copied from server to client and subsequently
  247. changed on both server and client,
  248. .I applychanges
  249. will not copy the client's new version to the server, because
  250. the server also has a new version.
  251. .I Applychanges
  252. and
  253. .I applylog
  254. detect the same conflicts; to resolve conflicts reported by
  255. .IR applychanges ,
  256. invoke
  257. .I applylog
  258. with the
  259. .B -c
  260. or
  261. .B -s
  262. flags.
  263. .SH EXAMPLE
  264. One might
  265. keep a client kfs file system up-to-date
  266. against a server file system using these tools.
  267. First, connect to a CPU server with a high-speed
  268. network connection to the file server and scan
  269. the server file system, updating the server database and log:
  270. .EX
  271. repl=$home/lib/replica
  272. proto=/sys/lib/sysconfig/proto/portproto
  273. db=$repl/srv.portproto.db
  274. log=$repl/srv.portproto.log
  275. 9fs $fs
  276. replica/updatedb -p $proto -r /n/$fs -x $repl $db >>$log
  277. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  278. .EE
  279. .PP
  280. Then, update the client file system:
  281. .EX
  282. repl=$home/lib/replica
  283. db=$repl/cli.portproto.db
  284. log=$repl/srv.portproto.log
  285. 9fs $fs
  286. 9fs kfs
  287. replica/applylog $db /n/kfs /n/$fs <$log
  288. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  289. .EE
  290. .PP
  291. The
  292. .B $repl
  293. directory is excluded from the sync so that multiple
  294. clients can each have their own local database.
  295. The shell scripts in
  296. .B /rc/bin/replica
  297. are essentially a further development of this example.
  298. .PP
  299. The Plan 9 distribution update program
  300. operates similarly, but omits the first scan;
  301. it is assumed that the Plan 9 developers run
  302. scans manually when the distribution
  303. file system changes.
  304. The manual page
  305. .IR replica (1)
  306. describes this in full.
  307. .SH SEE ALSO
  308. .IR replica (1)
  309. .SH BUGS
  310. These tools assume that
  311. .I mtime
  312. combined with
  313. .I length
  314. is a good indicator of changes to a file's contents.