#### ventiaux9.7 KB History Raw

 123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475 ``````.TH VENTIAUX 8 .SH NAME buildindex, checkarenas, checkindex, copy, fmtarenas, fmtindex, fmtisect, rdarena, rdarenablocks, read, wrarenablocks, write \- Venti maintenance and debugging commands .SH SYNOPSIS .B venti/buildindex [ .B -B .I blockcachesize ] [ .B -Z ] .I venti.config .I tmp .PP .B venti/checkarenas [ .B -afv ] .I file .PP .B venti/checkindex [ .B -f ] [ .B -B .I blockcachesize ] .I venti.config .I tmp .PP .B venti/copy [ .B -f ] .I src .I dst .I score [ .I type ] .PP .B venti/fmtarenas [ .B -Z ] [ .B -a .I arenasize ] [ .B -b .I blocksize ] .I name .I file .B venti/fmtindex [ .B -a ] .I venti.config .PP .B venti/fmtisect [ .B -Z ] [ .B -b .I blocksize ] .I name .I file .PP .B venti/rdarena [ .B -v ] .I arenapart .I arenaname .PP .B venti/read [ .B -h .I host ] .I score [ .I type ] .PP .B venti/wrarena [ .B -o .I fileoffset ] [ .B -h .I host ] .I arenafile [ .I clumpoffset ] .PP .B venti/write [ .B -h .I host ] [ .B -t .I type ] [ .B -z ] .SH DESCRIPTION These commands aid in the setup, maintenance, and debugging of Venti servers. See .IR venti (8) and .IR venti.conf (6) for an overview of the data structures stored by Venti. .PP Note that the units for the various sizes in the following commands can be specified by appending .LR k , .LR m , or .LR g to indicate kilobytes, megabytes, or gigabytes respectively. .PP .I Buildindex populates the index for the Venti system described in .IR venti.config . The index must have previously been formatted using .IR fmtindex . This command is typically used to build a new index for a Venti system when the old index becomes too small, or to rebuild an index after media failure. Small errors in an index can usually be fixed with .IR checkindex . .PP The .I tmp file, usually a disk partition, must be large enough to store a copy of the index. This temporary space is used to perform a merge sort of index entries generated by reading the arenas. .PP Options to .I buildindex are: .TP .BI -B " blockcachesize The amount of memory, in bytes, to use for caching raw disk accesses while running .IR buildindex . (This is not a property of the created index.) The default is 8k. .TP .B -Z Do not zero the index. This option should only be used when it is known that the index was already zeroed. .PD .PP .I Checkarenas examines the Venti arenas contained in the given .IR file . The program detects various error conditions, and optionally attempts to fix any errors that are found. .PP Options to .I checkarenas are: .TP .B -a For each arena, scan the entire data section. If this option is omitted, only the end section of the arena is examined. .TP .B -f Attempt to fix any errors that are found. .TP .B -v Increase the verbosity of output. .PD .PP .I Checkindex examines the Venti index described in .IR venti.config . The program detects various error conditions including: blocks that are not indexed, index entries for blocks that do not exist, and duplicate index entries. If requested, an attempt can be made to fix errors that are found. .PP The .I tmp file, usually a disk partition, must be large enough to store a copy of the index. This temporary space is used to perform a merge sort of index entries generated by reading the arenas. .PP Options to .I checkindex are: .TP .BI -B " blockcachesize The amount of memory, in bytes, to use for caching raw disk accesses while running .IR checkindex . The default is 8k. .TP .B -f Attempt to fix any errors that are found. .PD .PP .I Fmtarenas formats the given .IR file , typically a disk partition, into a number of Venti arenas. The arenas are given names of the form .IR name%d , where .I %d is replaced with a sequential number starting at 0. .PP Options to .I fmtarenas are: .TP .BI -a " arenasize The arenas are of .I arenasize bytes. The default is 512 megabytes, which was selected to provide a balance between the number of arenas and the ability to copy an arena to external media such as recordable CDs and tapes. .TP .BI -b " blocksize The size, in bytes, for read and write operations to the file. The size is recorded in the file, and is used by applications that access the arenas. The default is 8k. .TP .B -Z Do not zero the data sections of the arenas. Using this option reduces the formatting time but should only be used when it is known that the file was already zeroed. .PD .I Fmtindex takes the .IR venti.conf (6) file .I venti.config and initializes the index sections to form a usable index structure. The arena files and index sections must have previously been formatted using .I fmtarenas and .I fmtisect respectively. .PP The function of a Venti index is to map a SHA1 fingerprint to a location in the data section of one of the arenas. The index is composed of blocks, each of which contains the mapping for a fixed range of possible fingerprint values. .I Fmtindex determines the mapping between SHA1 values and the blocks of the collection of index sections. Once this mapping has been determined, it cannot be changed without rebuilding the index. The basic assumption in the current implementation is that the index structure is sufficiently empty that individual blocks of the index will rarely overflow. The total size of the index should be about 2% to 10% of the total size of the arenas, but the exact depends both the index block size and the compressed size of block stored to Venti. .PP .I Fmtindex also computes a mapping between a linear address space and the data section of the collection of arenas. The .B -a option can be used to add additional arenas to an index. To use this feature, add the new arenas to .I venti.config after the existing arenas and then run .I fmtindex .BR -a . .PP A copy of the above mappings is stored in the header for each of the index sections. These copies enable .I buildindex to restore a single index section without rebuilding the entire index. .PP .I Fmtisect formats the given .IR file , typically a disk partition, as a Venti index section with the specified .IR name . One or more formatted index sections are combined into a Venti index using .IR fmtindex . Each of the index sections within an index must have a unique name. .PP Options to .I fmtisect are: .TP .BI -b " blocksize The size, in bytes, for read and write operations to the file. All the index sections within a index must have the same block size. The default is 8k. .TP .B -Z Do not zero the index. Using this option reduces the formatting time but should only be used when it is known that the file was already zeroed. .PD .PP .I Rdarena extracts the named .I arena from the Venti system described by the .IR venti.conf (6) file .I venti.config and writes this arena to standard output. This command is typically used to back up an arena to external media. The .B -v option generates more verbose output on standard error. .PP .I Wrarena writes the blocks contained in the arena .I arenafile (typically, the output of .IR rdarena ) to a Venti server. It is typically used to reinitialize a Venti server from backups of the arenas. For example, .IP .EX venti/rdarena /dev/sdC0/arenas arena.0 >external.media venti/wrarena -h venti2 external.media .EE .LP writes the blocks contained in .B arena.0 to the Venti server .B venti2 (typically not the one using .BR /dev/sdC0/arenas ). .PP The .B -o option specifies that the arena starts at byte .I fileoffset (default .BR 0 ) in .I arenafile . This is useful for reading directly from the Venti arena partition: .IP .EX venti/wrarena -h venti2 -o 335872 /dev/sdC0/arenas .EE .LP (In this example, 335872 is the offset shown in the Venti server's index list (344064) minus one block (8192). You will need to substitute your own arena offsets and block size.) .PP Finally, the optional .I offset argument specifies that the writing should begin with the clump starting at .I offset within the arena. .I Wrarena prints the offset it stopped at (because there were no more data blocks). This could be used to incrementally back up a Venti server to another Venti server: .IP .EX last=`{cat last} venti/wrarena -h venti2 -o 335872 /dev/sdC0/arenas \$last >output awk '/^end offset/ { print \$3 }' offset >last .EE .LP Of course, one would need to add wrapper code to keep track of which arenas have been processed. See .B /sys/src/cmd/venti/backup.example for a version that does this. .PP .I Read and .I write read and write blocks from a running Venti server. They are intended to ease debugging of the server. The default .I host is the environment variable .BR \$venti , followed by the network metaname .BR \$venti . The .I type is the decimal type of block to be read or written. If no .I type is specified for .I read , all types are tried, and a command-line is printed to show the type that eventually worked. If no .I type is specified for .I write , .B VtDataType (13) is used. .I Read reads the block named by .I score (a SHA1 hash) from the Venti server and writes it to standard output. .I Write reads a block from standard input and attempts to write it to the Venti server. If successful, it prints the score of the block on the server. .PP .I Copy walks the entire tree of blocks rooted at .I score , copying all the blocks visited during the walk from the Venti server at network address .I src to the Venti server at network address .I dst . If .I type (a decimal block type for .IR score ) is omitted, all types will be tried in sequence until one is found that works. The .B -f flag runs the copy in ``fast'' mode: if a block is already on .IR dst , the walk does not descend below it, on the assumption that all its children are also already on .IR dst . Without this flag, the copy often transfers many times more data than necessary. .SH SOURCE .B /sys/src/cmd/venti .SH "SEE ALSO" .IR venti (8), .IR venti.conf (6) .SH BUGS .I Buildindex should allow an individual index section to be rebuilt. The merge sort could be performed in the space used to store the index rather than requiring a temporary file. ``````