2
0

README.ENGINE 16 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287
  1. ENGINE
  2. ======
  3. With OpenSSL 0.9.6, a new component was added to support alternative
  4. cryptography implementations, most commonly for interfacing with external
  5. crypto devices (eg. accelerator cards). This component is called ENGINE,
  6. and its presence in OpenSSL 0.9.6 (and subsequent bug-fix releases)
  7. caused a little confusion as 0.9.6** releases were rolled in two
  8. versions, a "standard" and an "engine" version. In development for 0.9.7,
  9. the ENGINE code has been merged into the main branch and will be present
  10. in the standard releases from 0.9.7 forwards.
  11. There are currently built-in ENGINE implementations for the following
  12. crypto devices:
  13. o Microsoft CryptoAPI
  14. o VIA Padlock
  15. o nCipher CHIL
  16. In addition, dynamic binding to external ENGINE implementations is now
  17. provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE"
  18. section below for details.
  19. At this stage, a number of things are still needed and are being worked on:
  20. 1 Integration of EVP support.
  21. 2 Configuration support.
  22. 3 Documentation!
  23. 1 With respect to EVP, this relates to support for ciphers and digests in
  24. the ENGINE model so that alternative implementations of existing
  25. algorithms/modes (or previously unimplemented ones) can be provided by
  26. ENGINE implementations.
  27. 2 Configuration support currently exists in the ENGINE API itself, in the
  28. form of "control commands". These allow an application to expose to the
  29. user/admin the set of commands and parameter types a given ENGINE
  30. implementation supports, and for an application to directly feed string
  31. based input to those ENGINEs, in the form of name-value pairs. This is an
  32. extensible way for ENGINEs to define their own "configuration" mechanisms
  33. that are specific to a given ENGINE (eg. for a particular hardware
  34. device) but that should be consistent across *all* OpenSSL-based
  35. applications when they use that ENGINE. Work is in progress (or at least
  36. in planning) for supporting these control commands from the CONF (or
  37. NCONF) code so that applications using OpenSSL's existing configuration
  38. file format can have ENGINE settings specified in much the same way.
  39. Presently however, applications must use the ENGINE API itself to provide
  40. such functionality. To see first hand the types of commands available
  41. with the various compiled-in ENGINEs (see further down for dynamic
  42. ENGINEs), use the "engine" openssl utility with full verbosity, ie;
  43. openssl engine -vvvv
  44. 3 Documentation? Volunteers welcome! The source code is reasonably well
  45. self-documenting, but some summaries and usage instructions are needed -
  46. moreover, they are needed in the same POD format the existing OpenSSL
  47. documentation is provided in. Any complete or incomplete contributions
  48. would help make this happen.
  49. STABILITY & BUG-REPORTS
  50. =======================
  51. What already exists is fairly stable as far as it has been tested, but
  52. the test base has been a bit small most of the time. For the most part,
  53. the vendors of the devices these ENGINEs support have contributed to the
  54. development and/or testing of the implementations, and *usually* (with no
  55. guarantees) have experience in using the ENGINE support to drive their
  56. devices from common OpenSSL-based applications. Bugs and/or inexplicable
  57. behaviour in using a specific ENGINE implementation should be sent to the
  58. author of that implementation (if it is mentioned in the corresponding C
  59. file), and in the case of implementations for commercial hardware
  60. devices, also through whatever vendor support channels are available. If
  61. none of this is possible, or the problem seems to be something about the
  62. ENGINE API itself (ie. not necessarily specific to a particular ENGINE
  63. implementation) then you should mail complete details to the relevant
  64. OpenSSL mailing list. For a definition of "complete details", refer to
  65. the OpenSSL "README" file. As for which list to send it to;
  66. openssl-users: if you are *using* the ENGINE abstraction, either in an
  67. pre-compiled application or in your own application code.
  68. openssl-dev: if you are discussing problems with OpenSSL source code.
  69. USAGE
  70. =====
  71. The default "openssl" ENGINE is always chosen when performing crypto
  72. operations unless you specify otherwise. You must actively tell the
  73. openssl utility commands to use anything else through a new command line
  74. switch called "-engine". Also, if you want to use the ENGINE support in
  75. your own code to do something similar, you must likewise explicitly
  76. select the ENGINE implementation you want.
  77. Depending on the type of hardware, system, and configuration, "settings"
  78. may need to be applied to an ENGINE for it to function as expected/hoped.
  79. The recommended way of doing this is for the application to support
  80. ENGINE "control commands" so that each ENGINE implementation can provide
  81. whatever configuration primitives it might require and the application
  82. can allow the user/admin (and thus the hardware vendor's support desk
  83. also) to provide any such input directly to the ENGINE implementation.
  84. This way, applications do not need to know anything specific to any
  85. device, they only need to provide the means to carry such user/admin
  86. input through to the ENGINE in question. Ie. this connects *you* (and
  87. your helpdesk) to the specific ENGINE implementation (and device), and
  88. allows application authors to not get buried in hassle supporting
  89. arbitrary devices they know (and care) nothing about.
  90. A new "openssl" utility, "openssl engine", has been added in that allows
  91. for testing and examination of ENGINE implementations. Basic usage
  92. instructions are available by specifying the "-?" command line switch.
  93. DYNAMIC ENGINES
  94. ===============
  95. The new "dynamic" ENGINE provides a low-overhead way to support ENGINE
  96. implementations that aren't pre-compiled and linked into OpenSSL-based
  97. applications. This could be because existing compiled-in implementations
  98. have known problems and you wish to use a newer version with an existing
  99. application. It could equally be because the application (or OpenSSL
  100. library) you are using simply doesn't have support for the ENGINE you
  101. wish to use, and the ENGINE provider (eg. hardware vendor) is providing
  102. you with a self-contained implementation in the form of a shared-library.
  103. The other use-case for "dynamic" is with applications that wish to
  104. maintain the smallest foot-print possible and so do not link in various
  105. ENGINE implementations from OpenSSL, but instead leaves you to provide
  106. them, if you want them, in the form of "dynamic"-loadable
  107. shared-libraries. It should be possible for hardware vendors to provide
  108. their own shared-libraries to support arbitrary hardware to work with
  109. applications based on OpenSSL 0.9.7 or later. If you're using an
  110. application based on 0.9.7 (or later) and the support you desire is only
  111. announced for versions later than the one you need, ask the vendor to
  112. backport their ENGINE to the version you need.
  113. How does "dynamic" work?
  114. ------------------------
  115. The dynamic ENGINE has a special flag in its implementation such that
  116. every time application code asks for the 'dynamic' ENGINE, it in fact
  117. gets its own copy of it. As such, multi-threaded code (or code that
  118. multiplexes multiple uses of 'dynamic' in a single application in any
  119. way at all) does not get confused by 'dynamic' being used to do many
  120. independent things. Other ENGINEs typically don't do this so there is
  121. only ever 1 ENGINE structure of its type (and reference counts are used
  122. to keep order). The dynamic ENGINE itself provides absolutely no
  123. cryptographic functionality, and any attempt to "initialise" the ENGINE
  124. automatically fails. All it does provide are a few "control commands"
  125. that can be used to control how it will load an external ENGINE
  126. implementation from a shared-library. To see these control commands,
  127. use the command-line;
  128. openssl engine -vvvv dynamic
  129. The "SO_PATH" control command should be used to identify the
  130. shared-library that contains the ENGINE implementation, and "NO_VCHECK"
  131. might possibly be useful if there is a minor version conflict and you
  132. (or a vendor helpdesk) is convinced you can safely ignore it.
  133. "ID" is probably only needed if a shared-library implements
  134. multiple ENGINEs, but if you know the engine id you expect to be using,
  135. it doesn't hurt to specify it (and this provides a sanity check if
  136. nothing else). "LIST_ADD" is only required if you actually wish the
  137. loaded ENGINE to be discoverable by application code later on using the
  138. ENGINE's "id". For most applications, this isn't necessary - but some
  139. application authors may have nifty reasons for using it. The "LOAD"
  140. command is the only one that takes no parameters and is the command
  141. that uses the settings from any previous commands to actually *load*
  142. the shared-library ENGINE implementation. If this command succeeds, the
  143. (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE
  144. that has been loaded from the shared-library. As such, any control
  145. commands supported by the loaded ENGINE could then be executed as per
  146. normal. Eg. if ENGINE "foo" is implemented in the shared-library
  147. "libfoo.so" and it supports some special control command "CMD_FOO", the
  148. following code would load and use it (NB: obviously this code has no
  149. error checking);
  150. ENGINE *e = ENGINE_by_id("dynamic");
  151. ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
  152. ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
  153. ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
  154. ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);
  155. For testing, the "openssl engine" utility can be useful for this sort
  156. of thing. For example the above code excerpt would achieve much the
  157. same result as;
  158. openssl engine dynamic \
  159. -pre SO_PATH:/lib/libfoo.so \
  160. -pre ID:foo \
  161. -pre LOAD \
  162. -pre "CMD_FOO:some input data"
  163. Or to simply see the list of commands supported by the "foo" ENGINE;
  164. openssl engine -vvvv dynamic \
  165. -pre SO_PATH:/lib/libfoo.so \
  166. -pre ID:foo \
  167. -pre LOAD
  168. Applications that support the ENGINE API and more specifically, the
  169. "control commands" mechanism, will provide some way for you to pass
  170. such commands through to ENGINEs. As such, you would select "dynamic"
  171. as the ENGINE to use, and the parameters/commands you pass would
  172. control the *actual* ENGINE used. Each command is actually a name-value
  173. pair and the value can sometimes be omitted (eg. the "LOAD" command).
  174. Whilst the syntax demonstrated in "openssl engine" uses a colon to
  175. separate the command name from the value, applications may provide
  176. their own syntax for making that separation (eg. a win32 registry
  177. key-value pair may be used by some applications). The reason for the
  178. "-pre" syntax in the "openssl engine" utility is that some commands
  179. might be issued to an ENGINE *after* it has been initialised for use.
  180. Eg. if an ENGINE implementation requires a smart-card to be inserted
  181. during initialisation (or a PIN to be typed, or whatever), there may be
  182. a control command you can issue afterwards to "forget" the smart-card
  183. so that additional initialisation is no longer possible. In
  184. applications such as web-servers, where potentially volatile code may
  185. run on the same host system, this may provide some arguable security
  186. value. In such a case, the command would be passed to the ENGINE after
  187. it has been initialised for use, and so the "-post" switch would be
  188. used instead. Applications may provide a different syntax for
  189. supporting this distinction, and some may simply not provide it at all
  190. ("-pre" is almost always what you're after, in reality).
  191. How do I build a "dynamic" ENGINE?
  192. ----------------------------------
  193. This question is trickier - currently OpenSSL bundles various ENGINE
  194. implementations that are statically built in, and any application that
  195. calls the "ENGINE_load_builtin_engines()" function will automatically
  196. have all such ENGINEs available (and occupying memory). Applications
  197. that don't call that function have no ENGINEs available like that and
  198. would have to use "dynamic" to load any such ENGINE - but on the other
  199. hand such applications would only have the memory footprint of any
  200. ENGINEs explicitly loaded using user/admin provided control commands.
  201. The main advantage of not statically linking ENGINEs and only using
  202. "dynamic" for hardware support is that any installation using no
  203. "external" ENGINE suffers no unnecessary memory footprint from unused
  204. ENGINEs. Likewise, installations that do require an ENGINE incur the
  205. overheads from only *that* ENGINE once it has been loaded.
  206. Sounds good? Maybe, but currently building an ENGINE implementation as
  207. a shared-library that can be loaded by "dynamic" isn't automated in
  208. OpenSSL's build process. It can be done manually quite easily however.
  209. Such a shared-library can either be built with any OpenSSL code it
  210. needs statically linked in, or it can link dynamically against OpenSSL
  211. if OpenSSL itself is built as a shared library. The instructions are
  212. the same in each case, but in the former (statically linked any
  213. dependencies on OpenSSL) you must ensure OpenSSL is built with
  214. position-independent code ("PIC"). The default OpenSSL compilation may
  215. already specify the relevant flags to do this, but you should consult
  216. with your compiler documentation if you are in any doubt.
  217. This example will show building the "atalla" ENGINE in the
  218. crypto/engine/ directory as a shared-library for use via the "dynamic"
  219. ENGINE.
  220. 1) "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL
  221. source tree.
  222. 2) Recompile at least one source file so you can see all the compiler
  223. flags (and syntax) being used to build normally. Eg;
  224. touch hw_atalla.c ; make
  225. will rebuild "hw_atalla.o" using all such flags.
  226. 3) Manually enter the same compilation line to compile the
  227. "hw_atalla.c" file but with the following two changes;
  228. (a) add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
  229. (b) change the output file from "hw_atalla.o" to something new,
  230. eg. "tmp_atalla.o"
  231. 4) Link "tmp_atalla.o" into a shared-library using the top-level
  232. OpenSSL libraries to resolve any dependencies. The syntax for doing
  233. this depends heavily on your system/compiler and is a nightmare
  234. known well to anyone who has worked with shared-library portability
  235. before. 'gcc' on Linux, for example, would use the following syntax;
  236. gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
  237. 5) Test your shared library using "openssl engine" as explained in the
  238. previous section. Eg. from the top-level directory, you might try;
  239. apps/openssl engine -vvvv dynamic \
  240. -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
  241. If the shared-library loads successfully, you will see both "-pre"
  242. commands marked as "SUCCESS" and the list of control commands
  243. displayed (because of "-vvvv") will be the control commands for the
  244. *atalla* ENGINE (ie. *not* the 'dynamic' ENGINE). You can also add
  245. the "-t" switch to the utility if you want it to try and initialise
  246. the atalla ENGINE for use to test any possible hardware/driver
  247. issues.
  248. PROBLEMS
  249. ========
  250. It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32.
  251. A quick test done right before the release showed that trying "openssl speed
  252. -engine cswift" generated errors. If the DSO gets enabled, an attempt is made
  253. to write at memory address 0x00000002.