123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153 |
- _ _ ____ _
- ___| | | | _ \| |
- / __| | | | |_) | |
- | (__| |_| | _ <| |___
- \___|\___/|_| \_\_____|
- The cURL Test Suite
- Requires:
- perl (and a unix-style shell)
- diff (when a test fails, a diff is shown)
- stunnel (for HTTPS and FTPS tests)
- OpenSSH or SunSSH (for SCP, SFTP and SOCKS4/5 tests)
- TCP ports used by default:
- - 8990 on localhost for HTTP tests
- - 8991 on localhost for HTTPS tests
- - 8994 on localhost for HTTP IPv6 tests
- - 8992 on localhost for FTP tests
- - 8995 on localhost for FTP (2) tests
- - 8993 on localhost for FTPS tests
- - 8996 on localhost for FTP IPv6 tests
- - 8997 on localhost for TFTP tests
- - 8999 on localhost for SCP/SFTP tests
- - 9000 on localhost for SOCKS tests
- The test suite runs simple FTP, HTTP and TFTP servers on these ports to
- which it makes requests. For SSL tests, it runs stunnel to handle
- encryption to the regular servers. For SSH, it runs a standard OpenSSH
- server. For SOCKS4/5 tests SSH is used to perform the SOCKS functionality
- and requires a SSH client and server.
- The base port number shown above can be changed using runtests' -b option
- to allow running more than one instance of the test suite simultaneously
- on one machine.
- Run:
- 'make test'. This builds the test suite support code and invokes the
- 'runtests.pl' perl script to run all the tests. Edit the top variables
- of that script in case you have some specific needs, or run the script
- manually (after the support code has been built).
- The script breaks on the first test that doesn't do OK. Use -a to prevent
- the script from abort on the first error. Run the script with -v for more
- verbose output. Use -d to run the test servers with debug output enabled as
- well. Specifying -k keeps all the log files generated by the test intact.
- Use -s for shorter output, or pass test numbers to run specific tests only
- (like "./runtests.pl 3 4" to test 3 and 4 only). It also supports test case
- ranges with 'to', as in "./runtests 3 to 9" which runs the seven tests from
- 3 to 9. Any test numbers starting with ! are disabled, as are any test
- numbers found in the file data/DISABLED (one per line).
- Shell startup scripts:
- Tests which use the ssh test server, SCP/SFTP/SOCKS tests, might be badly
- influenced by the output of system wide or user specific shell startup scripts,
- .bashrc, .profile, /etc/csh.cshrc, .login, /etc/bashrc, etc. which output text
- messages or escape sequences on user login. When these shell startup messages
- or escape sequences are output they might corrupt the expected stream of data
- which flows to the sftp-server or from the ssh client which can result in bad
- test behaviour or even prevent the test server from running.
- If the test suite ssh or sftp server fails to start up and logs the message
- 'Received message too long' then you are certainly suffering the unwanted
- output of a shell startup script. Locate, cleanup or adjust the shell script.
- Memory:
- The test script will check that all allocated memory is freed properly IF
- curl has been built with the CURLDEBUG define set. The script will
- automatically detect if that is the case, and it will use the ../memanalyze
- script to analyze the memory debugging output.
- The -t option will enable torture testing mode, which runs each test
- many times but causes a different memory allocation to fail on each
- successive run. This tests the out of memory error handling code to
- ensure that memory leaks do not occur even in those situations.
- Debug:
- If a test case fails, you can conveniently get the script to invoke the
- debugger (gdb) for you with the server running and the exact same command
- line parameters that failed. Just invoke 'runtests.pl <test number> -g' and
- then just type 'run' in the debugger to perform the command through the
- debugger.
- If a test case causes a core dump, analyze it by running gdb like:
- # gdb ../curl/src core
- ... and get a stack trace with the gdb command:
- (gdb) where
- Logs:
- All logs are generated in the logs/ subdirectory (it is emptied first
- in the runtests.pl script). Use runtests.pl -k to keep the temporary files
- after the test run.
- Data:
- All test cases are put in the data/ subdirectory. Each test is stored in the
- file named according to the test number.
- See FILEFORMAT for the description of the test case files.
- Code coverage:
- gcc provides a tool that can determine the code coverage figures for
- the test suite. To use it, configure curl with
- CFLAGS='-fprofile-arcs -ftest-coverage -g -O0'. Make sure you run the normal
- and torture tests to get more full coverage, i.e. do:
- make test
- make test-torture
- The graphical tool ggcov can be used to browse the source and create
- coverage reports on *NIX hosts:
- ggcov -r lib src
- The text mode tool gcov may also be used, but it doesn't handle object files
- in more than one directory very well.
- Remote testing:
- The runtests.pl script provides some hooks to allow curl to be tested on a
- machine where perl can not be run. The test framework in this case runs on
- a workstation where perl is available, while curl itself is run on a remote
- system using ssh or some other remote execution method. See the comments at
- the beginning of runtests.pl for details.
- TEST CASE NUMBERS
- So far, I've used this system:
- 1 - 99 HTTP
- 100 - 199 FTP*
- 200 - 299 FILE*
- 300 - 399 HTTPS
- 400 - 499 FTPS
- 500 - 599 libcurl source code tests, not using the curl command tool
- 600 - 699 SCP/SFTP
- 700 - 799 SOCKS4 (even numbers) and SOCK5 (odd numbers)
- 1000 - 1999 miscellaneous*
- 2000 - x multiple sequential protocols per test case*
- Since 30-apr-2003, there's nothing in the system that requires us to keep
- within these number series, and those sections marked with * actually
- contain tests for a variety of protocols. Each test case now specifies
- its own server requirements, independent of test number.
- TODO:
- * Add tests for TELNET, LDAP, DICT...
- * SOCKS4/5 test deficiencies - no proxy authentication tests as SSH (the
- test mechanism) doesn't support them
|