2
0

FILEFORMAT 17 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388389390391392393394395396397398399400401402403404405406407408409410411412413414415416417418419420421422423424425426427428429430431432433434435436437438439440441442443444445446447448449450451452453454455456457458459460461462463464465466467468469470471472473474475476477478
  1. The test suite's file format is very simple and extensible, closely
  2. resembling XML. All data for a single test case resides in a single
  3. ASCII file. Labels mark the beginning and the end of all sections, and each
  4. label must be written in its own line. Comments are either XML-style
  5. (enclosed with <!-- and -->) or C-style (beginning with #) and must appear
  6. on their own lines and not alongside actual test data. Most test data files
  7. are syntactically valid XML, although a few files are not (lack of
  8. support for character entities and the preservation of CR/LF characters at
  9. the end of lines are the biggest differences).
  10. The file begins with a 'testcase' tag, which encompasses the remainder of
  11. the file.
  12. <testcase>
  13. Each file is split up in three main sections: reply, client and verify. The
  14. reply section is used for the server to know what to send as a reply for the
  15. requests curl sends, the client section defines how the client should behave
  16. while the verify section defines how to verify that the data stored after a
  17. command has been run ended up correctly.
  18. Each main section has a number of available subsections that can be
  19. specified, that will be checked/used if specified. This document includes all
  20. the subsections currently supported.
  21. Main sections are 'info', 'reply', 'client' and 'verify'.
  22. <info>
  23. <keywords>
  24. A newline-separated list of keywords describing what this test case uses and
  25. tests. Try to use an already used keyword. These keywords will be used for
  26. statistical/informational purposes and for choosing or skipping classes
  27. of tests. "Keywords" must begin with an alphabetic character, "-", "["
  28. or "{" and may actually consist of multiple words separated by spaces
  29. which are treated together as a single identifier.
  30. </keywords>
  31. </info>
  32. <reply>
  33. <data [nocheck="yes"] [sendzero="yes"] [base64="yes"]>
  34. data to be sent to the client on its request and later verified that it arrived
  35. safely. Set nocheck="yes" to prevent the test script from verifying the arrival
  36. of this data.
  37. If the data contains 'swsclose' anywhere within the start and end tag, and
  38. this is a HTTP test, then the connection will be closed by the server after
  39. this response is sent. If not, the connection will be kept persistent.
  40. If the data contains 'swsbounce' anywhere within the start and end tag, the
  41. HTTP server will detect if this is a second request using the same test and
  42. part number and will then increase the part number with one. This is useful
  43. for auth tests and similar.
  44. 'sendzero' set to yes means that the (FTP) server will "send" the data even if
  45. the size is zero bytes. Used to verify curl's behaviour on zero bytes
  46. transfers.
  47. 'base64' set to yes means that the data provided in the test-file is a chunk
  48. of data encoded with base64. It is the only way a test case can contain binary
  49. data. (This attribute can in fact be used on any section, but it doesn't make
  50. much sense for other sections than "data").
  51. For FTP file listings, the <data> section will be used *only* if you make sure
  52. that there has been a CWD done first to a directory named 'test-[num]' where
  53. [num] is the test case number. Otherwise the ftp server can't know from which
  54. test file to load the list content.
  55. </data>
  56. <dataNUM>
  57. Send back this contents instead of the <data> one. The num is set by:
  58. A) The test number in the request line is >10000 and this is the remainder
  59. of [test case number]%10000.
  60. B) The request was HTTP and included digest details, which adds 1000 to NUM
  61. C) If a HTTP request is NTLM type-1, it adds 1001 to num
  62. D) If a HTTP request is NTLM type-3, it adds 1002 to num
  63. E) If a HTTP request is Basic and num is already >=1000, it adds 1 to num
  64. F) If a HTTP request is Negotiate, num gets incremented by one for each
  65. request with Negotiate authorization header on the same test case.
  66. Dynamically changing num in this way allows the test harness to be used to
  67. test authentication negotiation where several different requests must be sent
  68. to complete a transfer. The response to each request is found in its own data
  69. section. Validating the entire negotiation sequence can be done by
  70. specifying a datacheck section.
  71. </dataNUM>
  72. <connect>
  73. The connect section is used instead of the 'data' for all CONNECT
  74. requests. The remainder of the rules for the data section then apply but with
  75. a connect prefix.
  76. </connect>
  77. <datacheck [mode="text"] [nonewline="yes"]>
  78. if the data is sent but this is what should be checked afterwards. If
  79. 'nonewline' is set, we will cut off the trailing newline of this given data
  80. before comparing with the one actually received by the client.
  81. Use the mode="text" attribute if the output is in text mode on platforms that
  82. have a text/binary difference.
  83. </datacheck>
  84. <datacheckNUM [nonewline="yes"] [mode="text"]>
  85. The contents of numbered datacheck sections are appended to the non-numbered
  86. one.
  87. </datacheckNUM>
  88. <size>
  89. number to return on a ftp SIZE command (set to -1 to make this command fail)
  90. </size>
  91. <mdtm>
  92. what to send back if the client sends a (FTP) MDTM command, set to -1 to
  93. have it return that the file doesn't exist
  94. </mdtm>
  95. <postcmd>
  96. special purpose server-command to control its behavior *after* the
  97. reply is sent
  98. For HTTP/HTTPS, these are supported:
  99. wait [secs]
  100. - Pause for the given time
  101. </postcmd>
  102. <servercmd>
  103. Special-commands for the server.
  104. For FTP/SMTP/POP/IMAP, these are supported:
  105. REPLY [command] [return value] [response string]
  106. - Changes how the server responds to the [command]. [response string] is
  107. evaluated as a perl string, so it can contain embedded \r\n, for example.
  108. There's a special [command] named "welcome" (without quotes) which is the
  109. string sent immediately on connect as a welcome.
  110. REPLYLF (like above but sends the response terminated with LF-only and not
  111. CRLF)
  112. COUNT [command] [num]
  113. - Do the REPLY change for [command] only [num] times and then go back to the
  114. built-in approach
  115. DELAY [command] [secs]
  116. - Delay responding to this command for the given time
  117. RETRWEIRDO
  118. - Enable the "weirdo" RETR case when multiple response lines appear at once
  119. when a file is transferred
  120. RETRNOSIZE
  121. - Make sure the RETR response doesn't contain the size of the file
  122. NOSAVE
  123. - Don't actually save what is received
  124. SLOWDOWN
  125. - Send FTP responses with 0.01 sec delay between each byte
  126. PASVBADIP
  127. - makes PASV send back an illegal IP in its 227 response
  128. CAPA [capabilities]
  129. - Enables support for and specifies a list of space separated capabilities to
  130. return to the client for the IMAP CAPABILITY, POP3 CAPA and SMTP EHLO
  131. commands
  132. AUTH [mechanisms]
  133. - Enables support for SASL authentication and specifies a list of space
  134. separated mechanisms for IMAP, POP3 and SMTP
  135. For HTTP/HTTPS:
  136. auth_required if this is set and a POST/PUT is made without auth, the
  137. server will NOT wait for the full request body to get sent
  138. idle do nothing after receiving the request, just "sit idle"
  139. stream continuously send data to the client, never-ending
  140. writedelay: [secs] delay this amount between reply packets
  141. skip: [num] instructs the server to ignore reading this many bytes from a PUT
  142. or POST request
  143. rtp: part [num] channel [num] size [num]
  144. stream a fake RTP packet for the given part on a chosen channel
  145. with the given payload size
  146. connection-monitor When used, this will log [DISCONNECT] to the server.input
  147. log when the connection is disconnected.
  148. upgrade when an HTTP upgrade header is found, the server will upgrade
  149. to http2
  150. swsclose instruct server to close connection after response
  151. For TFTP:
  152. writedelay: [secs] delay this amount between reply packets (each packet being
  153. 512 bytes payload)
  154. </servercmd>
  155. </reply>
  156. <client>
  157. <server>
  158. What server(s) this test case requires/uses:
  159. file
  160. ftp
  161. ftp-ipv6
  162. ftps
  163. http
  164. http-ipv6
  165. http-proxy
  166. http-unix
  167. https
  168. httptls+srp
  169. httptls+srp-ipv6
  170. http/2
  171. imap
  172. none
  173. pop3
  174. rtsp
  175. rtsp-ipv6
  176. scp
  177. sftp
  178. smtp
  179. socks4
  180. socks5
  181. Give only one per line. This subsection is mandatory.
  182. </server>
  183. <features>
  184. A list of features that MUST be present in the client/library for this test to
  185. be able to run. If a required feature is not present then the test will be
  186. SKIPPED.
  187. Alternatively a feature can be prefixed with an exclamation mark to indicate a
  188. feature is NOT required. If the feature is present then the test will be
  189. SKIPPED.
  190. Features testable here are:
  191. crypto
  192. debug
  193. getrlimit
  194. GnuTLS
  195. GSS-API
  196. http/2
  197. idn
  198. ipv6
  199. Kerberos
  200. large_file
  201. libz
  202. manual
  203. Metalink
  204. NSS
  205. NTLM
  206. OpenSSL
  207. PSL
  208. socks
  209. SPNEGO
  210. SSL
  211. SSLpinning
  212. SSPI
  213. TLS-SRP
  214. TrackMemory
  215. threaded-resolver
  216. unittest
  217. unix-sockets
  218. WinSSL
  219. ld_preload
  220. alt-svc
  221. as well as each protocol that curl supports. A protocol only needs to be
  222. specified if it is different from the server (useful when the server
  223. is 'none').
  224. </features>
  225. <killserver>
  226. Using the same syntax as in <server> but when mentioned here these servers
  227. are explicitly KILLED when this test case is completed. Only use this if there
  228. is no other alternatives. Using this of course requires subsequent tests to
  229. restart servers.
  230. </killserver>
  231. <precheck>
  232. A command line that if set gets run by the test script before the test. If an
  233. output is displayed by the command or if the return code is non-zero, the test
  234. will be skipped and the (single-line) output will be displayed as reason for
  235. not running the test. Variables are substituted as in the <command> section.
  236. </precheck>
  237. <postcheck>
  238. A command line that if set gets run by the test script after the test. If
  239. the command exists with a non-zero status code, the test will be considered
  240. to have failed. Variables are substituted as in the <command> section.
  241. </postcheck>
  242. <tool>
  243. Name of tool to use instead of "curl". This tool must be built and exist
  244. either in the libtest/ directory (if the tool starts with 'lib') or in the
  245. unit/ directory (if the tool starts with 'unit').
  246. </tool>
  247. <name>
  248. test case description
  249. </name>
  250. <setenv>
  251. variable1=contents1
  252. variable2=contents2
  253. Set the given environment variables to the specified value before the actual
  254. command is run. They are cleared again after the command has been run.
  255. Variables are first substituted as in the <command> section.
  256. </setenv>
  257. <command [option="no-output/no-include/force-output"] [timeout="secs"]
  258. [delay="secs"][type="perl"]>
  259. command line to run, there's a bunch of %variables that get replaced
  260. accordingly.
  261. Note that the URL that gets passed to the server actually controls what data
  262. that is returned. The last slash in the URL must be followed by a number. That
  263. number (N) will be used by the test-server to load test case N and return the
  264. data that is defined within the <reply><data></data></reply> section.
  265. If there's no test number found above, the HTTP test server will use the
  266. number following the last dot in the given hostname (made so that a CONNECT
  267. can still pass on test number) so that "foo.bar.123" gets treated as test case
  268. 123. Alternatively, if an IPv6 address is provided to CONNECT, the last
  269. hexadecimal group in the address will be used as the test number! For example
  270. the address "[1234::ff]" would be treated as test case 255.
  271. Set type="perl" to write the test case as a perl script. It implies that
  272. there's no memory debugging and valgrind gets shut off for this test.
  273. Set option="no-output" to prevent the test script to slap on the --output
  274. argument that directs the output to a file. The --output is also not added if
  275. the verify/stdout section is used.
  276. Set option="force-output" to make use of --output even when the test is
  277. otherwise written to verify stdout.
  278. Set option="no-include" to prevent the test script to slap on the --include
  279. argument.
  280. Set timeout="secs" to override default server logs advisor read lock timeout.
  281. This timeout is used by the test harness, once that the command has completed
  282. execution, to wait for the test server to write out server side log files and
  283. remove the lock that advised not to read them. The "secs" parameter is the not
  284. negative integer number of seconds for the timeout. This 'timeout' attribute
  285. is documented for completeness sake, but is deep test harness stuff and only
  286. needed for very singular and specific test cases. Avoid using it.
  287. Set delay="secs" to introduce a time delay once that the command has completed
  288. execution and before the <postcheck> section runs. The "secs" parameter is the
  289. not negative integer number of seconds for the delay. This 'delay' attribute
  290. is intended for very specific test cases, and normally not needed.
  291. Available substitute variables include:
  292. %CLIENT6IP - IPv6 address of the client running curl
  293. %CLIENTIP - IPv4 address of the client running curl
  294. %CURL - Path to the curl executable
  295. %FTP2PORT - Port number of the FTP server 2
  296. %FTP6PORT - IPv6 port number of the FTP server
  297. %FTPPORT - Port number of the FTP server
  298. %FTPSPORT - Port number of the FTPS server
  299. %FTPTIME2 - Timeout in seconds that should be just sufficient to receive
  300. a response from the test FTP server
  301. %FTPTIME3 - Even longer than %FTPTIME2
  302. %GOPHER6PORT - IPv6 port number of the Gopher server
  303. %GOPHERPORT - Port number of the Gopher server
  304. %HOST6IP - IPv6 address of the host running this test
  305. %HOSTIP - IPv4 address of the host running this test
  306. %HTTP6PORT - IPv6 port number of the HTTP server
  307. %HTTPUNIXPATH - Path to the Unix socket of the HTTP server
  308. %HTTPPORT - Port number of the HTTP server
  309. %HTTPSPORT - Port number of the HTTPS server
  310. %HTTPTLS6PORT - IPv6 port number of the HTTP TLS server
  311. %HTTPTLSPORT - Port number of the HTTP TLS server
  312. %IMAP6PORT - IPv6 port number of the IMAP server
  313. %IMAPPORT - Port number of the IMAP server
  314. %POP36PORT - IPv6 port number of the POP3 server
  315. %POP3PORT - Port number of the POP3 server
  316. %PROXYPORT - Port number of the HTTP proxy
  317. %PWD - Current directory
  318. %POSIX_PWD - Current directory somewhat mingw friendly
  319. %FILE_PWD - Current directory, on windows prefixed with a slash
  320. %RTSP6PORT - IPv6 port number of the RTSP server
  321. %RTSPPORT - Port number of the RTSP server
  322. %SMTP6PORT - IPv6 port number of the SMTP server
  323. %SMTPPORT - Port number of the SMTP server
  324. %SOCKSPORT - Port number of the SOCKS4/5 server
  325. %SRCDIR - Full path to the source dir
  326. %SSHPORT - Port number of the SCP/SFTP server
  327. %TFTP6PORT - IPv6 port number of the TFTP server
  328. %TFTPPORT - Port number of the TFTP server
  329. %USER - Login ID of the user running the test
  330. </command>
  331. <file name="log/filename">
  332. This creates the named file with this content before the test case is run,
  333. which is useful if the test case needs a file to act on.
  334. Variables are substituted on the contents of the file as in the <command>
  335. section.
  336. </file>
  337. <stdin [nonewline="yes"]>
  338. Pass this given data on stdin to the tool.
  339. If 'nonewline' is set, we will cut off the trailing newline of this given data
  340. before comparing with the one actually received by the client
  341. </stdin>
  342. </client>
  343. <verify>
  344. <errorcode>
  345. numerical error code curl is supposed to return. Specify a list of accepted
  346. error codes by separating multiple numbers with comma. See test 237 for an
  347. example.
  348. </errorcode>
  349. <strip>
  350. One regex per line that is removed from the protocol dumps before the
  351. comparison is made. This is very useful to remove dependencies on dynamically
  352. changing protocol data such as port numbers or user-agent strings.
  353. </strip>
  354. <strippart>
  355. One perl op per line that operates on the protocol dump. This is pretty
  356. advanced. Example: "s/^EPRT .*/EPRT stripped/"
  357. </strippart>
  358. <protocol [nonewline="yes"]>
  359. the protocol dump curl should transmit, if 'nonewline' is set, we will cut off
  360. the trailing newline of this given data before comparing with the one actually
  361. sent by the client Variables are substituted as in the <command> section. The
  362. <strip> and <strippart> rules are applied before comparisons are made.
  363. </protocol>
  364. <proxy [nonewline="yes"]>
  365. The protocol dump curl should transmit to a HTTP proxy (when the http-proxy
  366. server is used), if 'nonewline' is set, we will cut off the trailing newline
  367. of this given data before comparing with the one actually sent by the client
  368. Variables are substituted as in the <command> section. The <strip> and
  369. <strippart> rules are applied before comparisons are made.
  370. </proxy>
  371. <stdout [mode="text"] [nonewline="yes"]>
  372. This verifies that this data was passed to stdout. Variables are
  373. substituted as in the <command> section.
  374. Use the mode="text" attribute if the output is in text mode on platforms that
  375. have a text/binary difference.
  376. If 'nonewline' is set, we will cut off the trailing newline of this given data
  377. before comparing with the one actually received by the client
  378. </stdout>
  379. <file name="log/filename" [mode="text"]>
  380. The file's contents must be identical to this after the test is complete.
  381. Use the mode="text" attribute if the output is in text mode on platforms that
  382. have a text/binary difference.
  383. Variables are substituted as in the <command> section.
  384. </file>
  385. <file1>
  386. 1 to 4 can be appended to 'file' to compare more files.
  387. </file1>
  388. <file2>
  389. </file2>
  390. <file3>
  391. </file3>
  392. <file4>
  393. </file4>
  394. <stripfile>
  395. One perl op per line that operates on the output file or stdout before being
  396. compared with what is stored in the test file. This is pretty
  397. advanced. Example: "s/^EPRT .*/EPRT stripped/"
  398. </stripfile>
  399. <stripfile1>
  400. 1 to 4 can be appended to 'stripfile' to strip the correspending <fileN>
  401. content
  402. </stripfile1>
  403. <stripfile2>
  404. </stripfile2>
  405. <stripfile3>
  406. </stripfile3>
  407. <stripfile4>
  408. </stripfile4>
  409. <upload>
  410. the contents of the upload data curl should have sent
  411. </upload>
  412. <valgrind>
  413. disable - disables the valgrind log check for this test
  414. </valgrind>
  415. </verify>
  416. </testcase>