FILEFORMAT 14 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379
  1. The test suite's file format is very simple and extensible, closely
  2. resembling XML. All data for a single test case resides in a single
  3. ASCII file. Labels mark the beginning and the end of all sections, and each
  4. label must be written in its own line. Comments are either XML-style
  5. (enclosed with <!-- and -->) or C-style (beginning with #) and must appear
  6. on their own lines and not alongside actual test data. Most test data files
  7. are syntactically valid XML, although a few files are not (lack of
  8. support for character entities and the preservation of CR/LF characters at
  9. the end of lines are the biggest differences).
  10. The file begins with a 'testcase' tag, which encompasses the remainder of
  11. the file.
  12. <testcase>
  13. Each file is split up in three main sections: reply, client and verify. The
  14. reply section is used for the server to know what to send as a reply for the
  15. requests curl sends, the client section defines how the client should behave
  16. while the verify section defines how to verify that the data stored after a
  17. command has been run ended up correctly.
  18. Each main section has a number of available subsections that can be
  19. specified, that will be checked/used if specified. This document includes all
  20. the subsections currently supported.
  21. Main sections are 'info', 'reply', 'client' and 'verify'.
  22. <info>
  23. <keywords>
  24. A newline-separated list of keywords describing what this test case uses and
  25. tests. Try to use an already used keyword. These keywords will be used for
  26. statistical/informational purposes and for choosing or skipping classes
  27. of tests. "Keywords" must begin with an alphabetic character, "-", "["
  28. or "{" and may actually consist of multiple words separated by spaces
  29. which are treated together as a single identifier.
  30. </keywords>
  31. </info>
  32. <reply>
  33. <data [nocheck="yes"] [sendzero="yes"] [base64="yes"]>
  34. data to be sent to the client on its request and later verified that it arrived
  35. safely. Set nocheck="yes" to prevent the test script from verifying the arrival
  36. of this data.
  37. If the data contains 'swsclose' anywhere within the start and end tag, and
  38. this is a HTTP test, then the connection will be closed by the server after
  39. this response is sent. If not, the connection will be kept persistent.
  40. If the data contains 'swsbounce' anywhere within the start and end tag, the
  41. HTTP server will detect if this is a second request using the same test and
  42. part number and will then increase the part number with one. This is useful
  43. for auth tests and similar.
  44. 'sendzero' set to yes means that the (FTP) server will "send" the data even if
  45. the size is zero bytes. Used to verify curl's behaviour on zero bytes
  46. transfers.
  47. 'base64' set to yes means that the data provided in the test-file is a chunk
  48. of data encoded with base64. It is the only way a test case can contain binary
  49. data. (This attribute can in fact be used on any section, but it doesn't make
  50. much sense for other sections than "data").
  51. </data>
  52. <dataNUM>
  53. Send back this contents instead of the <data> one. The num is set by:
  54. A) The test number in the request line is >10000 and this is the remainder
  55. of [test case number]%10000.
  56. B) The request was HTTP and included digest details, which adds 1000 to NUM
  57. C) If a HTTP request is NTLM type-1, it adds 1001 to num
  58. D) If a HTTP request is NTLM type-3, it adds 1002 to num
  59. E) If a HTTP request is Basic and num is already >=1000, it adds 1 to num
  60. Dynamically changing num in this way allows the test harness to be used to
  61. test authentication negotiation where several different requests must be sent
  62. to complete a transfer. The response to each request is found in its own data
  63. section. Validating the entire negotiation sequence can be done by
  64. specifying a datacheck section.
  65. </dataNUM>
  66. <datacheck [nonewline="yes"]>
  67. if the data is sent but this is what should be checked afterwards. If
  68. 'nonewline' is set, we will cut off the trailing newline of this given data
  69. before comparing with the one actually received by the client
  70. </datacheck>
  71. <size>
  72. number to return on a ftp SIZE command (set to -1 to make this command fail)
  73. </size>
  74. <mdtm>
  75. what to send back if the client sends a (FTP) MDTM command, set to -1 to
  76. have it return that the file doesn't exist
  77. </mdtm>
  78. <postcmd>
  79. special purpose server-command to control its behavior *after* the
  80. reply is sent
  81. For HTTP/HTTPS, these are supported:
  82. wait [secs]
  83. - Pause for the given time
  84. </postcmd>
  85. <servercmd>
  86. Special-commands for the server.
  87. For FTP/SMTP/POP/IMAP, these are supported:
  88. REPLY [command] [return value] [response string]
  89. - Changes how the server responds to the [command]. [response string] is
  90. evaluated as a perl string, so it can contain embedded \r\n, for example.
  91. COUNT [command] [num]
  92. - Do the REPLY change for [command] only [num] times and then go back to the
  93. built-in approach
  94. DELAY [command] [secs]
  95. - Delay responding to this command for the given time
  96. RETRWEIRDO
  97. - Enable the "weirdo" RETR case when multiple response lines appear at once
  98. when a file is transfered
  99. RETRNOSIZE
  100. - Make sure the RETR response doesn't contain the size of the file
  101. NOSAVE
  102. - Don't actually save what is received
  103. SLOWDOWN
  104. - Send FTP responses with 0.01 sec delay between each byte
  105. PASVBADIP
  106. - makes PASV send back an illegal IP in its 227 response
  107. For HTTP/HTTPS:
  108. auth_required if this is set and a POST/PUT is made without auth, the
  109. server will NOT wait for the full request body to get sent
  110. idle do nothing after receiving the request, just "sit idle"
  111. stream continuously send data to the client, never-ending
  112. writedelay: [secs] delay this amount between reply packets
  113. pipe: [num] tell the server to expect this many HTTP requests before
  114. sending back anything, to allow pipelining tests
  115. skip: [num] instructs the server to ignore reading this many bytes from a PUT
  116. or POST request
  117. rtp: part [num] channel [num] size [num]
  118. stream a fake RTP packet for the given part on a chosen channel
  119. with the given payload size
  120. connection-monitor When used, this will log [DISCONNECT] to the server.input
  121. log when the connection is disconnected.
  122. </servercmd>
  123. </reply>
  124. <client>
  125. <server>
  126. What server(s) this test case requires/uses:
  127. file
  128. ftp
  129. ftp-ipv6
  130. ftps
  131. http
  132. http-ipv6
  133. https
  134. none
  135. scp
  136. sftp
  137. socks4
  138. socks5
  139. rtsp
  140. rtsp-ipv6
  141. imap
  142. pop3
  143. smtp
  144. httptls+srp
  145. httptls+srp-ipv6
  146. http-proxy
  147. Give only one per line. This subsection is mandatory.
  148. </server>
  149. <features>
  150. A list of features that MUST be present in the client/library for this test to
  151. be able to run (if these features are not present, the test will be
  152. SKIPPED). Features testable here are:
  153. axTLS
  154. crypto
  155. getrlimit
  156. GnuTLS
  157. idn
  158. ipv6
  159. large_file
  160. libz
  161. NSS
  162. NTLM
  163. OpenSSL
  164. SSL
  165. socks
  166. unittest
  167. debug
  168. TLS-SRP
  169. Metalink
  170. as well as each protocol that curl supports. A protocol only needs to be
  171. specified if it is different from the server (useful when the server
  172. is 'none').
  173. </features>
  174. <killserver>
  175. Using the same syntax as in <server> but when mentioned here these servers
  176. are explicitly KILLED when this test case is completed. Only use this if there
  177. is no other alternatives. Using this of course requires subsequent tests to
  178. restart servers.
  179. </killserver>
  180. <precheck>
  181. A command line that if set gets run by the test script before the test. If an
  182. output is displayed by the command or if the return code is non-zero, the test
  183. will be skipped and the (single-line) output will be displayed as reason for
  184. not running the test. Variables are substituted as in the <command> section.
  185. </precheck>
  186. <postcheck>
  187. A command line that if set gets run by the test script after the test. If
  188. the command exists with a non-zero status code, the test will be considered
  189. to have failed. Variables are substituted as in the <command> section.
  190. </postcheck>
  191. <tool>
  192. Name of tool to use instead of "curl". This tool must be built and exist
  193. either in the libtest/ directory (if the tool starts with 'lib') or in the
  194. unit/ directory (if the tool starts with 'unit').
  195. </tool>
  196. <name>
  197. test case description
  198. </name>
  199. <setenv>
  200. variable1=contents1
  201. variable2=contents2
  202. Set the given environment variables to the specified value before the actual
  203. command is run. They are cleared again after the command has been run.
  204. Variables are first substituted as in the <command> section.
  205. </setenv>
  206. <command [option="no-output/no-include"] [timeout="secs"] [delay="secs"]
  207. [type="perl"]>
  208. command line to run, there's a bunch of %variables that get replaced
  209. accordingly.
  210. Note that the URL that gets passed to the server actually controls what data
  211. that is returned. The last slash in the URL must be followed by a number. That
  212. number (N) will be used by the test-server to load test case N and return the
  213. data that is defined within the <reply><data></data></reply> section.
  214. If a CONNECT is used to the server (to emulate HTTPS etc over proxy), the port
  215. number given in the CONNECT request will be used to identify which test that
  216. is being run, if the proxy host name is said to start with 'test'.
  217. Set type="perl" to write the test case as a perl script. It implies that
  218. there's no memory debugging and valgrind gets shut off for this test.
  219. Set option="no-output" to prevent the test script to slap on the --output
  220. argument that directs the output to a file. The --output is also not added if
  221. the verify/stdout section is used.
  222. Set option="no-include" to prevent the test script to slap on the --include
  223. argument.
  224. Set timeout="secs" to override default server logs advisor read lock timeout.
  225. This timeout is used by the test harness, once that the command has completed
  226. execution, to wait for the test server to write out server side log files and
  227. remove the lock that advised not to read them. The "secs" parameter is the not
  228. negative integer number of seconds for the timeout. This 'timeout' attribute
  229. is documented for completeness sake, but is deep test harness stuff and only
  230. needed for very singular and specific test cases. Avoid using it.
  231. Set delay="secs" to introduce a time delay once that the command has completed
  232. execution and before the <postcheck> section runs. The "secs" parameter is the
  233. not negative integer number of seconds for the delay. This 'delay' attribute
  234. is intended for very specific test cases, and normally not needed.
  235. Available substitute variables include:
  236. %CLIENTIP - IPv4 address of the client running curl
  237. %CLIENT6IP - IPv6 address of the client running curl
  238. %HOSTIP - IPv4 address of the host running this test
  239. %HTTPPORT - Port number of the HTTP server
  240. %HOST6IP - IPv6 address of the host running this test
  241. %HTTP6PORT - IPv6 port number of the HTTP server
  242. %HTTPSPORT - Port number of the HTTPS server
  243. %PROXYPORT - Port number of the HTTP proxy
  244. %FTPPORT - Port number of the FTP server
  245. %FTP6PORT - IPv6 port number of the FTP server
  246. %FTPSPORT - Port number of the FTPS server
  247. %FTP2PORT - Port number of the FTP server 2
  248. %FTPTIME2 - Timeout in seconds that should be just sufficient to receive
  249. a response from the test FTP server
  250. %TFTPPORT - Port number of the TFTP server
  251. %TFTP6PORT - IPv6 port number of the TFTP server
  252. %SSHPORT - Port number of the SCP/SFTP server
  253. %SOCKSPORT - Port number of the SOCKS4/5 server
  254. %RTSPPORT - Port number of the RTSP server
  255. %RTSP6PORT - IPv6 port number of the RTSP server
  256. %SRCDIR - Full path to the source dir
  257. %PWD - Current directory
  258. %CURL - Path to the curl executable
  259. %USER - Login ID of the user running the test
  260. </command>
  261. <file name="log/filename">
  262. This creates the named file with this content before the test case is run,
  263. which is useful if the test case needs a file to act on.
  264. Variables are substituted on the contents of the file as in the <command>
  265. section.
  266. </file>
  267. <stdin>
  268. Pass this given data on stdin to the tool.
  269. </stdin>
  270. </client>
  271. <verify>
  272. <errorcode>
  273. numerical error code curl is supposed to return. Specify a list of accepted
  274. error codes by separating multiple numbers with comma. See test 237 for an
  275. example.
  276. </errorcode>
  277. <strip>
  278. One regex per line that is removed from the protocol dumps before the
  279. comparison is made. This is very useful to remove dependencies on dynamically
  280. changing protocol data such as port numbers or user-agent strings.
  281. </strip>
  282. <strippart>
  283. One perl op per line that operates on the protocol dump. This is pretty
  284. advanced. Example: "s/^EPRT .*/EPRT stripped/"
  285. </strippart>
  286. <protocol [nonewline="yes"]>
  287. the protocol dump curl should transmit, if 'nonewline' is set, we will cut off
  288. the trailing newline of this given data before comparing with the one actually
  289. sent by the client Variables are substituted as in the <command> section. The
  290. <strip> and <strippart> rules are applied before comparisons are made.
  291. </protocol>
  292. <proxy [nonewline="yes"]>
  293. The protocol dump curl should transmit to a HTTP proxy (when the http-proxy
  294. server is used), if 'nonewline' is set, we will cut off the trailing newline
  295. of this given data before comparing with the one actually sent by the client
  296. Variables are substituted as in the <command> section. The <strip> and
  297. <strippart> rules are applied before comparisons are made.
  298. </proxy>
  299. <stdout [mode="text"] [nonewline="yes"]>
  300. This verifies that this data was passed to stdout. Variables are
  301. substituted as in the <command> section.
  302. Use the mode="text" attribute if the output is in text mode on platforms that
  303. have a text/binary difference.
  304. If 'nonewline' is set, we will cut off the trailing newline of this given data
  305. before comparing with the one actually received by the client
  306. </stdout>
  307. <file name="log/filename" [mode="text"]>
  308. The file's contents must be identical to this after the test is complete.
  309. Use the mode="text" attribute if the output is in text mode on platforms that
  310. have a text/binary difference.
  311. Variables are substituted as in the <command> section.
  312. </file>
  313. <stripfile>
  314. One perl op per line that operates on the file before being compared. This is
  315. pretty advanced. Example: "s/^EPRT .*/EPRT stripped/"
  316. </stripfile>
  317. <upload>
  318. the contents of the upload data curl should have sent
  319. </upload>
  320. <valgrind>
  321. disable - disables the valgrind log check for this test
  322. </valgrind>
  323. </verify>
  324. </testcase>