replica 6.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307
  1. .TH REPLICA 8
  2. .SH NAME
  3. applychanges, applylog, compactdb, updatedb \- simple client-server replica management
  4. .SH SYNOPSIS
  5. .B replica/compactdb
  6. .I db
  7. .br
  8. .B replica/updatedb
  9. [
  10. .B -cl
  11. ]
  12. [
  13. .B -p
  14. .I proto
  15. ]
  16. [
  17. .B -r
  18. .I root
  19. ]
  20. [
  21. .B -t
  22. .I now
  23. .I n
  24. ]
  25. [
  26. .B -u
  27. .I uid
  28. ]
  29. [
  30. .B -x
  31. .I path
  32. ] ...
  33. .I db
  34. .br
  35. .B replica/applylog
  36. [
  37. .B -cnsuv
  38. ]
  39. .I clientdb
  40. .I clientroot
  41. .I serverroot
  42. [
  43. .I path ...
  44. ]
  45. .br
  46. .B replica/applychanges
  47. [
  48. .B -nuv
  49. ]
  50. [
  51. .B -p
  52. .I proto
  53. ]
  54. [
  55. .B -x
  56. .I path
  57. ] ...
  58. .I clientdb
  59. .I clientroot
  60. .I serverroot
  61. [
  62. .I path ...
  63. ]
  64. .SH DESCRIPTION
  65. These four tools collectively provide simple log-based
  66. client-server replica management.
  67. The shell scripts described in
  68. .IR replica (1)
  69. provide a more polished interface.
  70. .PP
  71. Both client and server maintain textual databases of file system
  72. metadata. Each line is of the form
  73. .ift .sp 0.5
  74. .ifn .sp
  75. \h'0.25i'
  76. .I path
  77. .I mode
  78. .I uid
  79. .I gid
  80. .I mtime
  81. .I length
  82. .ift .sp 0.5
  83. .ifn .sp
  84. Later entries for a
  85. .I path
  86. supersede previous ones.
  87. A line with the string
  88. .B REMOVED
  89. in the
  90. .I mode
  91. field annuls all previous entries for that
  92. .IR path .
  93. The entries in a file are typically kept sorted by
  94. .I path
  95. but need not be.
  96. These properties facilitate updating the database atomically
  97. by appending to it.
  98. .I Compactdb
  99. reads in a database and writes out an equivalent one,
  100. sorted by path and without outdated or annulled records.
  101. .PP
  102. A replica is further described on the server by a textual
  103. log listing creation and deletion of files and changes
  104. to file contents and metadata.
  105. Each line is of the form:
  106. .ift .sp 0.5
  107. .ifn .sp
  108. \h'0.25i'
  109. .I time
  110. .I gen
  111. .I verb
  112. .I path
  113. .I serverpath
  114. .I mode
  115. .I uid
  116. .I gid
  117. .I mtime
  118. .I length
  119. .ift .sp 0.5
  120. .ifn .sp
  121. The
  122. .I time
  123. and
  124. .I gen
  125. fields are both decimal numbers, providing an ordering
  126. for log entries so that incremental tools need not process
  127. the whole log each time they are run.
  128. The
  129. .IR verb ,
  130. a single character,
  131. describes the event:
  132. addition of a file
  133. .RB ( a ),
  134. deletion of a file
  135. .RB ( d ),
  136. a change to a file's contents
  137. .RB ( c ),
  138. or a change to a file's metadata
  139. .RB ( m ).
  140. .I Path
  141. is the file path on the client;
  142. .I serverpath
  143. the path on the server (these are different when the
  144. optional fifth field in a proto file line is given;
  145. see
  146. .IR proto (2)).
  147. .IR Mode ,
  148. .IR uid ,
  149. .IR gid ,
  150. and
  151. .I mtime
  152. are the files metadata as in the
  153. .B Dir
  154. structure (see
  155. .IR stat (5)).
  156. For deletion events, the metadata is that of the deleted file.
  157. For other events, the metadata is that after the event.
  158. .PP
  159. .I Updatedb
  160. scans the file system rooted at
  161. .I root
  162. for changes not present in
  163. .IR db ,
  164. noting them by appending new entries to the database
  165. and by writing log events to standard output.
  166. The
  167. .B -c
  168. option causes
  169. .I updatedb
  170. to consider only file and metadata changes, ignoring file additions and deletions.
  171. By default, the log events have
  172. .I time
  173. set to the current system time
  174. and use incrementing
  175. .I gen
  176. numbers starting at 0.
  177. The
  178. .B -t
  179. option can be used to specify a different time and starting number.
  180. If the
  181. .B -u
  182. option is given, all database entries and log events will use
  183. .I uid
  184. rather than the actual uids.
  185. The
  186. .B -x
  187. option (which may be specified multiple times) excludes the named path
  188. and all its children from the scan.
  189. If the
  190. .B -l
  191. option is given, the database is not changed and the
  192. .I time
  193. and
  194. .I gen
  195. fields are omitted from the log events;
  196. the resulting output is intended to be a human-readable
  197. summary of file system activity since the last scan.
  198. .PP
  199. .I Applylog
  200. is used to propagate changes from server to client.
  201. It applies the changes listed in a log
  202. (read from standard input)
  203. to the file system rooted at
  204. .IR clientroot ,
  205. copying files when necessary from the file system rooted at
  206. .IR serverroot .
  207. By default,
  208. .I applylog
  209. does not attempt to set the uid on files; the
  210. .B -u
  211. flag enables this.
  212. .I Applylog
  213. will not overwrite local changes made to replicated files.
  214. When it detects such conflicts, by default it prints an error describing
  215. the conflict and takes no action.
  216. If the
  217. .B -c
  218. flag is given,
  219. .I applylog
  220. still takes no action, but does so silently and will not
  221. report the conflicts in the future.
  222. (The conflict is resolved in favor of the client.)
  223. If the
  224. .B -s
  225. flag is given,
  226. .I applylog
  227. overwrites the local changes.
  228. (The conflict is resolved in favor of the server.)
  229. .PP
  230. .I Applychanges
  231. is, in some sense, the opposite of
  232. .IR applylog ;
  233. it scans the client file system for changes, and applies
  234. those changes to the server file system.
  235. .I Applychanges
  236. will not overwrite remote changes made to replicated files.
  237. For example, if a file is copied from server to client and subsequently
  238. changed on both server and client,
  239. .I applychanges
  240. will not copy the client's new version to the server, because
  241. the server also has a new version.
  242. .I Applychanges
  243. and
  244. .I applylog
  245. detect the same conflicts; to resolve conflicts reported by
  246. .IR applychanges ,
  247. invoke
  248. .I applylog
  249. with the
  250. .B -c
  251. or
  252. .B -s
  253. flags.
  254. .SH EXAMPLE
  255. One might
  256. keep a client kfs file system up-to-date
  257. against a server file system using these tools.
  258. First, connect to a CPU server with a high-speed
  259. network connection to the file server and scan
  260. the server file system, updating the server database and log:
  261. .EX
  262. repl=$home/lib/replica
  263. proto=/sys/lib/sysconfig/proto/portproto
  264. db=$repl/srv.portproto.db
  265. log=$repl/srv.portproto.log
  266. 9fs $fs
  267. replica/updatedb -p $proto -r /n/$fs -x $repl $db >>$log
  268. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  269. .EE
  270. .PP
  271. Then, update the client file system:
  272. .EX
  273. repl=$home/lib/replica
  274. db=$repl/cli.portproto.db
  275. log=$repl/srv.portproto.log
  276. 9fs $fs
  277. 9fs kfs
  278. replica/applylog $db /n/kfs /n/$fs <$log
  279. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  280. .EE
  281. .PP
  282. The
  283. .B $repl
  284. directory is excluded from the sync so that multiple
  285. clients can each have their own local database.
  286. The shell scripts in
  287. .B /rc/bin/replica
  288. are essentially a further development of this example.
  289. .PP
  290. The Plan 9 distribution update program
  291. operates similarly, but omits the first scan;
  292. it is assumed that the Plan 9 developers run
  293. scans manually when the distribution
  294. file system changes.
  295. The manual page
  296. .IR replica (1)
  297. describes this in full.
  298. .SH SEE ALSO
  299. .IR replica (1)
  300. .SH BUGS
  301. These tools assume that
  302. .I mtime
  303. combined with
  304. .I length
  305. is a good indicator of changes to a file's contents.