README 4.4 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149
  1. How to add recipes
  2. ==================
  3. For any test that you want to perform, you write a script located in
  4. test/recipes/, named {nn}-test_{name}.t, where {nn} is a two digit number and
  5. {name} is a unique name of your choice.
  6. Please note that if a test involves a new testing executable, you will need to
  7. do some additions in test/Makefile. More on this later.
  8. Naming conventions
  9. =================
  10. A test executable is named test/{name}test.c
  11. A test recipe is named test/recipes/{nn}-test_{name}.t, where {nn} is a two
  12. digit number and {name} is a unique name of your choice.
  13. The number {nn} is (somewhat loosely) grouped as follows:
  14. 00-04 sanity, internal and essential API tests
  15. 05-09 individual symmetric cipher algorithms
  16. 10-14 math (bignum)
  17. 15-19 individual asymmetric cipher algorithms
  18. 20-24 openssl commands (some otherwise not tested)
  19. 25-29 certificate forms, generation and verification
  20. 30-35 engine and evp
  21. 60-79 APIs
  22. 70 PACKET layer
  23. 80-89 "larger" protocols (CA, CMS, OCSP, SSL, TSA)
  24. 90-98 misc
  25. 99 most time consuming tests [such as test_fuzz]
  26. A recipe that just runs a test executable
  27. =========================================
  28. A script that just runs a program looks like this:
  29. #! /usr/bin/perl
  30. use OpenSSL::Test::Simple;
  31. simple_test("test_{name}", "{name}test", "{name}");
  32. {name} is the unique name you have chosen for your test.
  33. The second argument to `simple_test' is the test executable, and `simple_test'
  34. expects it to be located in test/
  35. For documentation on OpenSSL::Test::Simple, do
  36. `perldoc util/perl/OpenSSL/Test/Simple.pm'.
  37. A recipe that runs a more complex test
  38. ======================================
  39. For more complex tests, you will need to read up on Test::More and
  40. OpenSSL::Test. Test::More is normally preinstalled, do `man Test::More' for
  41. documentation. For OpenSSL::Test, do `perldoc util/perl/OpenSSL/Test.pm'.
  42. A script to start from could be this:
  43. #! /usr/bin/perl
  44. use strict;
  45. use warnings;
  46. use OpenSSL::Test;
  47. setup("test_{name}");
  48. plan tests => 2; # The number of tests being performed
  49. ok(test1, "test1");
  50. ok(test2, "test1");
  51. sub test1
  52. {
  53. # test feature 1
  54. }
  55. sub test2
  56. {
  57. # test feature 2
  58. }
  59. Changes to test/build.info
  60. ==========================
  61. Whenever a new test involves a new test executable you need to do the
  62. following (at all times, replace {NAME} and {name} with the name of your
  63. test):
  64. * add {name} to the list of programs under PROGRAMS_NO_INST
  65. * create a three line description of how to build the test, you will have
  66. to modify the include paths and source files if you don't want to use the
  67. basic test framework:
  68. SOURCE[{name}]={name}.c
  69. INCLUDE[{name}]=.. ../include
  70. DEPEND[{name}]=../libcrypto libtestutil.a
  71. Generic form of C test executables
  72. ==================================
  73. #include "testutil.h"
  74. static int my_test(void)
  75. {
  76. int testresult = 0; /* Assume the test will fail */
  77. int observed;
  78. observed = function(); /* Call the code under test */
  79. if (!TEST_int_equal(observed, 2)) /* Check the result is correct */
  80. goto end; /* Exit on failure - optional */
  81. testresult = 1; /* Mark the test case a success */
  82. end:
  83. cleanup(); /* Any cleanup you require */
  84. return testresult;
  85. }
  86. int setup_tests(void)
  87. {
  88. ADD_TEST(my_test); /* Add each test separately */
  89. return 1; /* Indicate success */
  90. }
  91. You should use the TEST_xxx macros provided by testutil.h to test all failure
  92. conditions. These macros produce an error message in a standard format if the
  93. condition is not met (and nothing if the condition is met). Additional
  94. information can be presented with the TEST_info macro that takes a printf
  95. format string and arguments. TEST_error is useful for complicated conditions,
  96. it also takes a printf format string and argument. In all cases the TEST_xxx
  97. macros are guaranteed to evaluate their arguments exactly once. This means
  98. that expressions with side effects are allowed as parameters. Thus,
  99. if (!TEST_ptr(ptr = OPENSSL_malloc(..)))
  100. works fine and can be used in place of:
  101. ptr = OPENSSL_malloc(..);
  102. if (!TEST_ptr(ptr))
  103. The former produces a more meaningful message on failure than the latter.