TLS/SSL and crypto library https://www.openssl.org

Neil Horman 1967539e21 Fix threadstest wrapping again 3 days ago
.ctags.d 859521e579 util/ctags.sh: a script for generating tags file with expanding macros 10 months ago
.github 7394de95b6 Add 3.3 and 3.2 branches to Coveralls builds 3 days ago
Configurations aba6219346 Ensure `$(MAKE)` commands and `CFLAGS` are appropriately quoted in the Makefile. 1 month ago
VMS 3077bfb78e Fix VMS installation - update vmsconfig.pm for consistency 2 months ago
apps 6af739b79b apps/req,crl: exit with 1 on verification failure 2 days ago
cloudflare-quiche @ 7ab6a55cfe 1cfdbdd0d4 Update Cloudflare Quiche to fix a build issue 7 months ago
crypto 4a9e48f727 Correct OSSL_sleep for NonStop PUT model by introducing sleep(). 5 days ago
demos 87e747000f Replace size_t with int and add the check for the EVP_MD_get_size() 3 days ago
dev 4f41334b4a Add known issues section 4 months ago
doc 6af739b79b apps/req,crl: exit with 1 on verification failure 2 days ago
engines c91f0ca958 Fix dasync_rsa_decrypt to call EVP_PKEY_meth_get_decrypt 2 weeks ago
exporters dd5fe94a61 exporters/cmake/OpenSSLConfig.cmake.in: Detect missing `platform->sharedlib_import` 3 months ago
external 05c9c7b02d Update the bundled external perl module Text-Template to version 1.56 4 years ago
fuzz c5e097dec5 Add Content Type OID for id-ct-rpkiSignedPrefixList 1 week ago
gost-engine @ ede3886cc5 e7d5d61b66 Update gost-engine submodule to fix the CI 2 days ago
include c5e097dec5 Add Content Type OID for id-ct-rpkiSignedPrefixList 1 week ago
krb5 @ aa9b4a2a64 dd62ec2777 Update dependencies for krb5 external test 2 years ago
ms 0856bf519f uplink-x86_64.pl: make x86_64-xlate.pl accept $flavour parameter 2 months ago
oqs-provider @ 8c746d7e29 f3f3f86a14 updated (lib+)oqsprovider to latest releases 9 months ago
os-dep 9d987de3aa Fix copyrights 2 years ago
providers 347f05e893 riscv: Fix remaining asm checks 6 days ago
pyca-cryptography @ 7e33b0e773 a8e7bc7c74 Update pyca-cryptography submodule to fix CI 1 year ago
python-ecdsa @ 4de8d5bf89 cccbb4fa60 TLSfuzzer: submodules 2 years ago
ssl bc3eb7b527 Try to fix intermittent CI failures in quic_multistream test 1 week ago
test 1967539e21 Fix threadstest wrapping again 2 days ago
tlsfuzzer @ dbd56c1490 cccbb4fa60 TLSfuzzer: submodules 2 years ago
tlslite-ng @ 771e9f59d6 cccbb4fa60 TLSfuzzer: submodules 2 years ago
tools 706fc5f6eb c_rehash: Fix file extension matching 1 year ago
util 7f8aba2f44 Limit the number of http headers when receiving the http response 2 weeks ago
wycheproof @ 2196000605 a09fb26ba9 add wycheproof submodule 2 years ago
.git-blame-ignore-revs 1f03d33ef5 Add reformatting commit to .git-blame-ignore-revs 2 weeks ago
.gitattributes cede07dc51 Remove the external BoringSSL test 3 years ago
.gitignore c768ccebc7 Add exporters for CMake 4 months ago
.gitmodules 3565cc2c5b Add openssl/fuzz-corpora repository as submodule 11 months ago
ACKNOWLEDGEMENTS.md d7f3a2cc86 Fix various typos, repeated words, align some spelling to LDP. 1 year ago
AUTHORS.md af403db090 Add some missing committers to the AUTHORS list 3 years ago
CHANGES.md 6af739b79b apps/req,crl: exit with 1 on verification failure 2 days ago
CODE-OF-CONDUCT.md 63df86b041 Add CODE-OF-CONDUCT.md 1 year ago
CONTRIBUTING.md ea15508b4f CONTRIBUTING.md: add missing "on" 1 month ago
Configure e98940d6f6 Enable qlog support by default 3 weeks ago
HACKING.md af33b200da Fixed some grammar and spelling 1 year ago
INSTALL.md 53ef123f48 Update FIPS hmac key documentation 6 days ago
LICENSE.txt 036cbb6bbf Rename NOTES*, README*, VERSION, HACKING, LICENSE to .md or .txt 3 years ago
NEWS.md d8d1910761 Amend CHANGES.md/NEWS.md to reflect DN output changes 1 month ago
NOTES-ANDROID.md 2d321448b2 Add support in configuration for android-riscv64 1 month ago
NOTES-DJGPP.md 4148581eb2 Unify the markdown links to the NOTES and README files 3 years ago
NOTES-NONSTOP.md 99fb31c167 Add atexit configuration option to using atexit() in libcrypto at build-time. 1 month ago
NOTES-PERL.md d7f3a2cc86 Fix various typos, repeated words, align some spelling to LDP. 1 year ago
NOTES-UNIX.md af33b200da Fixed some grammar and spelling 1 year ago
NOTES-VALGRIND.md 44fde44193 changes opensssl typos to openssl 2 years ago
NOTES-VMS.md d500f04400 Add information on the 'ias' port for OpenVMS 10 months ago
NOTES-WINDOWS.md cf6342bc02 NOTES-WINDOWS: fix named anchor links in table of contents 4 months ago
README-ENGINES.md af33b200da Fixed some grammar and spelling 1 year ago
README-FIPS.md 2b42290f08 Add FIPS build instructions 8 months ago
README-PROVIDERS.md af33b200da Fixed some grammar and spelling 1 year ago
README-QUIC.md ada33e98f5 Add a separate README for the guide demos 5 months ago
README.md 4f0172c543 README: add link to OpenSSL 3.2 manual pages 4 months ago
SUPPORT.md 3410f1045a Fix Markdown links in SUPPORT.md 2 years ago
VERSION.dat 186b3f6a01 Prepare for 3.3 5 months ago
build.info c768ccebc7 Add exporters for CMake 4 months ago
config 2f44c8151e config: Turn into a simple wrapper 3 years ago
config.com e39e295e20 Update copyright year 3 years ago
configdata.pm.in aa2d7e0ee1 Use $config{build_file} instead of $target{build_file} 1 year ago

README-ENGINES.md

Engines

Deprecation Note

The ENGINE API was introduced in OpenSSL version 0.9.6 as a low level interface for adding alternative implementations of cryptographic primitives, most notably for integrating hardware crypto devices.

The ENGINE interface has its limitations and it has been superseded by the PROVIDER API, it is deprecated in OpenSSL version 3.0. The following documentation is retained as an aid for users who need to maintain or support existing ENGINE implementations. Support for new hardware devices or new algorithms should be added via providers, and existing engines should be converted to providers as soon as possible.

Built-in ENGINE implementations

There are currently built-in ENGINE implementations for the following crypto devices:

  • Microsoft CryptoAPI
  • VIA Padlock
  • nCipher CHIL

In addition, dynamic binding to external ENGINE implementations is now provided by a special ENGINE called "dynamic". See the "DYNAMIC ENGINE" section below for details.

At this stage, a number of things are still needed and are being worked on:

  1. Integration of EVP support.
  2. Configuration support.
  3. Documentation!

Integration of EVP support

With respect to EVP, this relates to support for ciphers and digests in the ENGINE model so that alternative implementations of existing algorithms/modes (or previously unimplemented ones) can be provided by ENGINE implementations.

Configuration support

Configuration support currently exists in the ENGINE API itself, in the form of "control commands". These allow an application to expose to the user/admin the set of commands and parameter types a given ENGINE implementation supports, and for an application to directly feed string based input to those ENGINEs, in the form of name-value pairs. This is an extensible way for ENGINEs to define their own "configuration" mechanisms that are specific to a given ENGINE (eg. for a particular hardware device) but that should be consistent across all OpenSSL-based applications when they use that ENGINE. Work is in progress (or at least in planning) for supporting these control commands from the CONF (or NCONF) code so that applications using OpenSSL's existing configuration file format can have ENGINE settings specified in much the same way. Presently however, applications must use the ENGINE API itself to provide such functionality. To see first hand the types of commands available with the various compiled-in ENGINEs (see further down for dynamic ENGINEs), use the "engine" openssl utility with full verbosity, i.e.:

openssl engine -vvvv

Documentation

Documentation? Volunteers welcome! The source code is reasonably well self-documenting, but some summaries and usage instructions are needed - moreover, they are needed in the same POD format the existing OpenSSL documentation is provided in. Any complete or incomplete contributions would help make this happen.

STABILITY & BUG-REPORTS

What already exists is fairly stable as far as it has been tested, but the test base has been a bit small most of the time. For the most part, the vendors of the devices these ENGINEs support have contributed to the development and/or testing of the implementations, and usually (with no guarantees) have experience in using the ENGINE support to drive their devices from common OpenSSL-based applications. Bugs and/or inexplicable behaviour in using a specific ENGINE implementation should be sent to the author of that implementation (if it is mentioned in the corresponding C file), and in the case of implementations for commercial hardware devices, also through whatever vendor support channels are available. If none of this is possible, or the problem seems to be something about the ENGINE API itself (ie. not necessarily specific to a particular ENGINE implementation) then you should mail complete details to the relevant OpenSSL mailing list. For a definition of "complete details", refer to the OpenSSL "README" file. As for which list to send it to:

  • openssl-users: if you are using the ENGINE abstraction, either in an pre-compiled application or in your own application code.

  • openssl-dev: if you are discussing problems with OpenSSL source code.

USAGE

The default "openssl" ENGINE is always chosen when performing crypto operations unless you specify otherwise. You must actively tell the openssl utility commands to use anything else through a new command line switch called "-engine". Also, if you want to use the ENGINE support in your own code to do something similar, you must likewise explicitly select the ENGINE implementation you want.

Depending on the type of hardware, system, and configuration, "settings" may need to be applied to an ENGINE for it to function as expected/hoped. The recommended way of doing this is for the application to support ENGINE "control commands" so that each ENGINE implementation can provide whatever configuration primitives it might require and the application can allow the user/admin (and thus the hardware vendor's support desk also) to provide any such input directly to the ENGINE implementation. This way, applications do not need to know anything specific to any device, they only need to provide the means to carry such user/admin input through to the ENGINE in question. Ie. this connects you (and your helpdesk) to the specific ENGINE implementation (and device), and allows application authors to not get buried in hassle supporting arbitrary devices they know (and care) nothing about.

A new "openssl" utility, "openssl engine", has been added in that allows for testing and examination of ENGINE implementations. Basic usage instructions are available by specifying the "-?" command line switch.

DYNAMIC ENGINES

The new "dynamic" ENGINE provides a low-overhead way to support ENGINE implementations that aren't pre-compiled and linked into OpenSSL-based applications. This could be because existing compiled-in implementations have known problems and you wish to use a newer version with an existing application. It could equally be because the application (or OpenSSL library) you are using simply doesn't have support for the ENGINE you wish to use, and the ENGINE provider (eg. hardware vendor) is providing you with a self-contained implementation in the form of a shared-library. The other use-case for "dynamic" is with applications that wish to maintain the smallest foot-print possible and so do not link in various ENGINE implementations from OpenSSL, but instead leaves you to provide them, if you want them, in the form of "dynamic"-loadable shared-libraries. It should be possible for hardware vendors to provide their own shared-libraries to support arbitrary hardware to work with applications based on OpenSSL 0.9.7 or later. If you're using an application based on 0.9.7 (or later) and the support you desire is only announced for versions later than the one you need, ask the vendor to backport their ENGINE to the version you need.

How does "dynamic" work?

The dynamic ENGINE has a special flag in its implementation such that every time application code asks for the 'dynamic' ENGINE, it in fact gets its own copy of it. As such, multi-threaded code (or code that multiplexes multiple uses of 'dynamic' in a single application in any way at all) does not get confused by 'dynamic' being used to do many independent things. Other ENGINEs typically don't do this so there is only ever 1 ENGINE structure of its type (and reference counts are used to keep order). The dynamic ENGINE itself provides absolutely no cryptographic functionality, and any attempt to "initialise" the ENGINE automatically fails. All it does provide are a few "control commands" that can be used to control how it will load an external ENGINE implementation from a shared-library. To see these control commands, use the command-line;

openssl engine -vvvv dynamic

The "SO_PATH" control command should be used to identify the shared-library that contains the ENGINE implementation, and "NO_VCHECK" might possibly be useful if there is a minor version conflict and you (or a vendor helpdesk) is convinced you can safely ignore it. "ID" is probably only needed if a shared-library implements multiple ENGINEs, but if you know the engine id you expect to be using, it doesn't hurt to specify it (and this provides a sanity check if nothing else). "LIST_ADD" is only required if you actually wish the loaded ENGINE to be discoverable by application code later on using the ENGINE's "id". For most applications, this isn't necessary - but some application authors may have nifty reasons for using it. The "LOAD" command is the only one that takes no parameters and is the command that uses the settings from any previous commands to actually load the shared-library ENGINE implementation. If this command succeeds, the (copy of the) 'dynamic' ENGINE will magically morph into the ENGINE that has been loaded from the shared-library. As such, any control commands supported by the loaded ENGINE could then be executed as per normal. For instance, if ENGINE "foo" is implemented in the shared-library "libfoo.so" and it supports some special control command "CMD_FOO", the following code would load and use it (NB: obviously this code has no error checking);

ENGINE *e = ENGINE_by_id("dynamic");
ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libfoo.so", 0);
ENGINE_ctrl_cmd_string(e, "ID", "foo", 0);
ENGINE_ctrl_cmd_string(e, "LOAD", NULL, 0);
ENGINE_ctrl_cmd_string(e, "CMD_FOO", "some input data", 0);

For testing, the "openssl engine" utility can be useful for this sort of thing. For example the above code excerpt would achieve much the same result as;

openssl engine dynamic \
          -pre SO_PATH:/lib/libfoo.so \
          -pre ID:foo \
          -pre LOAD \
          -pre "CMD_FOO:some input data"

Or to simply see the list of commands supported by the "foo" ENGINE;

openssl engine -vvvv dynamic \
          -pre SO_PATH:/lib/libfoo.so \
          -pre ID:foo \
          -pre LOAD

Applications that support the ENGINE API and more specifically, the "control commands" mechanism, will provide some way for you to pass such commands through to ENGINEs. As such, you would select "dynamic" as the ENGINE to use, and the parameters/commands you pass would control the actual ENGINE used. Each command is actually a name-value pair and the value can sometimes be omitted (eg. the "LOAD" command). Whilst the syntax demonstrated in "openssl engine" uses a colon to separate the command name from the value, applications may provide their own syntax for making that separation (eg. a win32 registry key-value pair may be used by some applications). The reason for the "-pre" syntax in the "openssl engine" utility is that some commands might be issued to an ENGINE after it has been initialised for use. Eg. if an ENGINE implementation requires a smart-card to be inserted during initialisation (or a PIN to be typed, or whatever), there may be a control command you can issue afterwards to "forget" the smart-card so that additional initialisation is no longer possible. In applications such as web-servers, where potentially volatile code may run on the same host system, this may provide some arguable security value. In such a case, the command would be passed to the ENGINE after it has been initialised for use, and so the "-post" switch would be used instead. Applications may provide a different syntax for supporting this distinction, and some may simply not provide it at all ("-pre" is almost always what you're after, in reality).

How do I build a "dynamic" ENGINE?

This question is trickier - currently OpenSSL bundles various ENGINE implementations that are statically built in, and any application that calls the "ENGINE_load_builtin_engines()" function will automatically have all such ENGINEs available (and occupying memory). Applications that don't call that function have no ENGINEs available like that and would have to use "dynamic" to load any such ENGINE - but on the other hand such applications would only have the memory footprint of any ENGINEs explicitly loaded using user/admin provided control commands. The main advantage of not statically linking ENGINEs and only using "dynamic" for hardware support is that any installation using no "external" ENGINE suffers no unnecessary memory footprint from unused ENGINEs. Likewise, installations that do require an ENGINE incur the overheads from only that ENGINE once it has been loaded.

Sounds good? Maybe, but currently building an ENGINE implementation as a shared-library that can be loaded by "dynamic" isn't automated in OpenSSL's build process. It can be done manually quite easily however. Such a shared-library can either be built with any OpenSSL code it needs statically linked in, or it can link dynamically against OpenSSL if OpenSSL itself is built as a shared library. The instructions are the same in each case, but in the former (statically linked any dependencies on OpenSSL) you must ensure OpenSSL is built with position-independent code ("PIC"). The default OpenSSL compilation may already specify the relevant flags to do this, but you should consult with your compiler documentation if you are in any doubt.

This example will show building the "atalla" ENGINE in the crypto/engine/ directory as a shared-library for use via the "dynamic" ENGINE.

  1. "cd" to the crypto/engine/ directory of a pre-compiled OpenSSL source tree.

  2. Recompile at least one source file so you can see all the compiler flags (and syntax) being used to build normally. Eg;

    touch hw_atalla.c ; make
    

    will rebuild "hw_atalla.o" using all such flags.

  3. Manually enter the same compilation line to compile the "hw_atalla.c" file but with the following two changes;

    • add "-DENGINE_DYNAMIC_SUPPORT" to the command line switches,
    • change the output file from "hw_atalla.o" to something new, eg. "tmp_atalla.o"
  4. Link "tmp_atalla.o" into a shared-library using the top-level OpenSSL libraries to resolve any dependencies. The syntax for doing this depends heavily on your system/compiler and is a nightmare known well to anyone who has worked with shared-library portability before. 'gcc' on Linux, for example, would use the following syntax;

    gcc -shared -o dyn_atalla.so tmp_atalla.o -L../.. -lcrypto
    
  5. Test your shared library using "openssl engine" as explained in the previous section. Eg. from the top-level directory, you might try

    apps/openssl engine -vvvv dynamic \
          -pre SO_PATH:./crypto/engine/dyn_atalla.so -pre LOAD
    

If the shared-library loads successfully, you will see both "-pre" commands marked as "SUCCESS" and the list of control commands displayed (because of "-vvvv") will be the control commands for the atalla ENGINE (ie. not the 'dynamic' ENGINE). You can also add the "-t" switch to the utility if you want it to try and initialise the atalla ENGINE for use to test any possible hardware/driver issues.

PROBLEMS

It seems like the ENGINE part doesn't work too well with CryptoSwift on Win32. A quick test done right before the release showed that trying "openssl speed -engine cswift" generated errors. If the DSO gets enabled, an attempt is made to write at memory address 0x00000002.