1
0

replica 6.1 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303
  1. .TH REPLICA 8
  2. .SH NAME
  3. applychanges, applylog, compactdb, updatedb \- simple client-server replica management
  4. .SH SYNOPSIS
  5. .B replica/compactdb
  6. .I db
  7. .br
  8. .B replica/updatedb
  9. [
  10. .B -cl
  11. ]
  12. [
  13. .B -p
  14. .I proto
  15. ]
  16. [
  17. .B -r
  18. .I root
  19. ]
  20. [
  21. .B -t
  22. .I now
  23. .I n
  24. ]
  25. [
  26. .B -u
  27. .I uid
  28. ]
  29. [
  30. .B -x
  31. .I path
  32. ] ...
  33. .I db
  34. .br
  35. .B replica/applylog
  36. [
  37. .B -cnsuv
  38. ]
  39. .I clientdb
  40. .I clientroot
  41. .I serverroot
  42. [
  43. .I path ...
  44. ]
  45. .br
  46. .B replica/applychanges
  47. [
  48. .B -nuv
  49. ]
  50. [
  51. .B -p
  52. .I proto
  53. ]
  54. [
  55. .B -x
  56. .I path
  57. ] ...
  58. .I clientdb
  59. .I clientroot
  60. .I serverroot
  61. [
  62. .I path ...
  63. ]
  64. .SH DESCRIPTION
  65. These four tools collectively provide simple log-based
  66. client-server replica management.
  67. The shell scripts described in
  68. .IR replica (1)
  69. provide a more polished interface.
  70. .PP
  71. Both client and server maintain textual databases of file system
  72. metadata. Each line is of the form
  73. .sp 0.5
  74. \h'0.25i'
  75. .I path
  76. .I mode
  77. .I uid
  78. .I gid
  79. .I mtime
  80. .I length
  81. .sp 0.5
  82. Later entries for a
  83. .I path
  84. supersede previous ones.
  85. A line with the string
  86. .B REMOVED
  87. in the
  88. .I mode
  89. field annuls all previous entries for that
  90. .IR path .
  91. The entries in a file are typically kept sorted by
  92. .I path
  93. but need not be.
  94. These properties facilitate updating the database atomically
  95. by appending to it.
  96. .I Compactdb
  97. reads in a database and writes out an equivalent one,
  98. sorted by path and without outdated or annulled records.
  99. .PP
  100. A replica is further described on the server by a textual
  101. log listing creation and deletion of files and changes
  102. to file contents and metadata.
  103. Each line is of the form:
  104. .sp 0.5
  105. \h'0.25i'
  106. .I time
  107. .I gen
  108. .I verb
  109. .I path
  110. .I serverpath
  111. .I mode
  112. .I uid
  113. .I gid
  114. .I mtime
  115. .I length
  116. .sp 0.5
  117. The
  118. .I time
  119. and
  120. .I gen
  121. fields are both decimal numbers, providing an ordering
  122. for log entries so that incremental tools need not process
  123. the whole log each time they are run.
  124. The
  125. .IR verb ,
  126. a single character,
  127. describes the event:
  128. addition of a file
  129. .RB ( a ),
  130. deletion of a file
  131. .RB ( d ),
  132. a change to a file's contents
  133. .RB ( c ),
  134. or a change to a file's metadata
  135. .RB ( m ).
  136. .I Path
  137. is the file path on the client;
  138. .I serverpath
  139. the path on the server (these are different when the
  140. optional fifth field in a proto file line is given;
  141. see
  142. .IR proto (2)).
  143. .IR Mode ,
  144. .IR uid ,
  145. .IR gid ,
  146. and
  147. .I mtime
  148. are the files metadata as in the
  149. .B Dir
  150. structure (see
  151. .IR stat (5)).
  152. For deletion events, the metadata is that of the deleted file.
  153. For other events, the metadata is that after the event.
  154. .PP
  155. .I Updatedb
  156. scans the file system rooted at
  157. .I root
  158. for changes not present in
  159. .IR db ,
  160. noting them by appending new entries to the database
  161. and by writing log events to standard output.
  162. The
  163. .B -c
  164. option causes
  165. .I updatedb
  166. to consider only file and metadata changes, ignoring file additions and deletions.
  167. By default, the log events have
  168. .I time
  169. set to the current system time
  170. and use incrementing
  171. .I gen
  172. numbers starting at 0.
  173. The
  174. .B -t
  175. option can be used to specify a different time and starting number.
  176. If the
  177. .B -u
  178. option is given, all database entries and log events will use
  179. .I uid
  180. rather than the actual uids.
  181. The
  182. .B -x
  183. option (which may be specified multiple times) excludes the named path
  184. and all its children from the scan.
  185. If the
  186. .B -l
  187. option is given, the database is not changed and the
  188. .I time
  189. and
  190. .I gen
  191. fields are omitted from the log events;
  192. the resulting output is intended to be a human-readable
  193. summary of file system activity since the last scan.
  194. .PP
  195. .I Applylog
  196. is used to propagate changes from server to client.
  197. It applies the changes listed in a log
  198. (read from standard input)
  199. to the file system rooted at
  200. .IR clientroot ,
  201. copying files when necessary from the file system rooted at
  202. .IR serverroot .
  203. By default,
  204. .I applylog
  205. does not attempt to set the uid on files; the
  206. .B -u
  207. flag enables this.
  208. .I Applylog
  209. will not overwrite local changes made to replicated files.
  210. When it detects such conflicts, by default it prints an error describing
  211. the conflict and takes no action.
  212. If the
  213. .B -c
  214. flag is given,
  215. .I applylog
  216. still takes no action, but does so silently and will not
  217. report the conflicts in the future.
  218. (The conflict is resolved in favor of the client.)
  219. If the
  220. .B -s
  221. flag is given,
  222. .I applylog
  223. overwrites the local changes.
  224. (The conflict is resolved in favor of the server.)
  225. .PP
  226. .I Applychanges
  227. is, in some sense, the opposite of
  228. .IR applylog ;
  229. it scans the client file system for changes, and applies
  230. those changes to the server file system.
  231. .I Applychanges
  232. will not overwrite remote changes made to replicated files.
  233. For example, if a file is copied from server to client and subsequently
  234. changed on both server and client,
  235. .I applychanges
  236. will not copy the client's new version to the server, because
  237. the server also has a new version.
  238. .I Applychanges
  239. and
  240. .I applylog
  241. detect the same conflicts; to resolve conflicts reported by
  242. .IR applychanges ,
  243. invoke
  244. .I applylog
  245. with the
  246. .B -c
  247. or
  248. .B -s
  249. flags.
  250. .SH EXAMPLE
  251. One might
  252. keep a client kfs file system up-to-date
  253. against a server file system using these tools.
  254. First, connect to a CPU server with a high-speed
  255. network connection to the file server and scan
  256. the server file system, updating the server database and log:
  257. .EX
  258. repl=$home/lib/replica
  259. proto=/sys/lib/sysconfig/proto/portproto
  260. db=$repl/srv.portproto.db
  261. log=$repl/srv.portproto.log
  262. 9fs $fs
  263. replica/updatedb -p $proto -r /n/$fs -x $repl $db >>$log
  264. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  265. .EE
  266. .PP
  267. Then, update the client file system:
  268. .EX
  269. repl=$home/lib/replica
  270. db=$repl/cli.portproto.db
  271. log=$repl/srv.portproto.log
  272. 9fs $fs
  273. 9fs kfs
  274. replica/applylog $db /n/kfs /n/$fs <$log
  275. replica/compactdb $db >/tmp/a && mv /tmp/a $db
  276. .EE
  277. .PP
  278. The
  279. .B $repl
  280. directory is excluded from the sync so that multiple
  281. clients can each have their own local database.
  282. The shell scripts in
  283. .B /rc/bin/replica
  284. are essentially a further development of this example.
  285. .PP
  286. The Plan 9 distribution update program
  287. operates similarly, but omits the first scan;
  288. it is assumed that the Plan 9 developers run
  289. scans manually when the distribution
  290. file system changes.
  291. The manual page
  292. .IR replica (1)
  293. describes this in full.
  294. .SH SEE ALSO
  295. .IR replica (1)
  296. .SH BUGS
  297. These tools assume that
  298. .I mtime
  299. combined with
  300. .I length
  301. is a good indicator of changes to a file's contents.