openssl/doc/designs/ddd/REPORT.md

Report on the Conclusions of the QUIC DDD Process

The QUIC Demo-Driven Design process was undertaken to meet the OMC requirement to develop a QUIC API that required only minimal changes to existing applications to be able to adapt their code to use QUIC. The demo-driven design process developed a set of representative demos modelling a variety of common OpenSSL usage patterns based on analysis of a broad spectrum of open source software projects using OpenSSL.

As part of this process, a set of proposed diffs were produced. These proposed diffs were the expected changes which would be needed to the baseline demos to support QUIC based on theoretical analysis of the minimum requirements to be able to support QUIC. This analysis concluded that the changes needed to applications could be kept very small in many circumstances, with only minimal diff sizes to the baseline demos.

Following the development of QUIC MVP, these demos have been revisited and the correspondence of our actual final API and usage patterns with the planned diffs have been reviewed.

This document discusses the planned changes and the actual changes for each demo and draws conclusions on the level of disparity.

Since tracking a set of diffs separately is unwieldy, both the planned and unplanned changes have been folded into the original baseline demo files guarded with #ifdef USE_QUIC. Viewing these files therefore is informative to application writers as it provides a clear view of what is different when using QUIC. (The originally planned changes, and the final changes, are added in separate, clearly-labelled commits; to view the originally planned changes only, view the commit history for a given demo file.)

ddd-01-conn-blocking

This demo exists to demonstrate the simplest possible usage of OpenSSL, whether with TLS or QUIC.

Originally planned changes

The originally planned change to enable applications for QUIC amounted to just a single line:

+    ctx = SSL_CTX_new(QUIC_client_method());
-    ctx = SSL_CTX_new(TLS_client_method());

Actual changes

The following additional changes needed to be made:

• QUIC_client_method was renamed to OSSL_QUIC_client_method for namespacing reasons.

• A call to SSL_set_alpn_protos to configure ALPN was added. This is necessary because QUIC mandates the use of ALPN, and this was not noted during the DDD process.

ddd-02-conn-nonblocking

This demo exists to demonstrate simple non-blocking usage. As with ddd-01-conn-blocking, the name resolution process is managed by BIO_s_connect.

It also arbitrarily adds a BIO_f_buffer pushed onto the BIO stack as this is a common application usage pattern.

Originally planned changes

The originally planned changes to enable applications for QUIC amounted to:

• Change of method (as for ddd-01-conn-blocking);

• Use of a BIO_f_dgram_buffer BIO method instead of a BIO_f_buffer;

• Use of a BIO_get_poll_fd function to get the FD to poll rather than BIO_get_fd;

• A change to how the POLLIN/POLLOUT/POLLERR flags to pass to poll(2) need to be determined.

• Additional functions in application code to determine event handling timeouts related to QUIC (get_conn_pump_timeout) and to pump the QUIC event loop (pump).

• Timeout computation code which involves merging and comparing different timeouts and calling pump as needed, based on deadlines reported by libssl.

Note that some of these changes are unnecessary when using the thread assisted mode (see the variant ddd-02-conn-nonblocking-threads below).

Actual changes

The following additional changes needed to be made:

• Change of method name (as for ddd-01-conn-blocking);

• Use of ALPN (as for ddd-01-conn-blocking);

• The strategy for how to expose pollable OS resource handles to applications to determine I/O readiness has changed substantially since the original DDD process. As such, applications now use BIO_get_rpoll_descriptor and BIO_get_wpoll_descriptor to determine I/O readiness, rather than the originally hypothesised SSL_get_poll_fd.

• The strategy for how to determine when to poll for POLLIN, when to poll for POLLOUT, etc. has changed since the original DDD process. This information is now exposed via SSL_net_read_desired and SSL_net_write_desired.

• The API to expose the event handling deadline for the QUIC engine has evolved since the original DDD process. The new API SSL_get_event_timeout is used, rather than the originally hypothesised BIO_get_timeout/SSL_get_timeout.

• The API to perform QUIC event processing has been renamed to be more descriptive. It is now called SSL_handle_events rather than the originally hypothesised BIO_pump/SSL_pump.

The following changes were foreseen to be necessary, but turned out to actually not be necessary:

• The need to change code which pushes a BIO_f_buffer() after a SSL BIO was foreseen as use of buffering on the network side is unworkable with QUIC. This turned out not to be necessary since we can just reject the BIO_push() call. The buffer should still be freed eventually when the SSL BIO is freed. The buffer is not used and is unnecessary, so it is still desirable for applications to remove this code.

ddd-02-conn-nonblocking-threads

This is a variant of the ddd-02-conn-nonblocking demo. The base is the same, but the changes made are different. The use of thread-assisted mode, in which an internal assist thread is used to perform QUIC event handling, enables an application to make fewer changes than are needed in the ddd-02-conn-nonblocking demo.

Originally planned changes

The originally planned changes to enable applications for QUIC amounted to:

• Change of method, this time using method QUIC_client_thread_method rather than QUIC_client_method;

• Use of a BIO_get_poll_fd function to get the FD to poll rather than BIO_get_fd;

• A change to how the POLLIN/POLLOUT/POLLERR flags to pass to poll(2) need to be determined.

Note that this is a substantially smaller list of changes than for ddd-02-conn-nonblocking.

Actual changes

The following additional changes needed to be made:

• Change of method name (QUIC_client_thread_method was renamed to OSSL_QUIC_client_thread_method for namespacing reasons);

• Use of ALPN (as for ddd-01-conn-blocking);

• Use of BIO_get_rpoll_descriptor rather than BIO_get_poll_fd (as for ddd-02-conn-nonblocking).

• Use of SSL_net_read_desired and SSL_net_write_desired (as for ddd-02-conn-nonblocking).

ddd-03-fd-blocking

This demo is similar to ddd-01-conn-blocking but uses a file descriptor passed directly by the application rather than BIO_s_connect.

Originally planned changes

• Change of method (as for ddd-01-conn-blocking);

• The arguments to the socket(2) call are changed from (AF_INET, SOCK_STREAM, IPPROTO_TCP) to (AF_INET, SOCK_DGRAM, IPPROTO_UDP).

Actual changes

The following additional changes needed to be made:

• Change of method name (as for ddd-01-conn-blocking);

• Use of ALPN (as for ddd-01-conn-blocking).

ddd-04-fd-nonblocking

This demo is similar to ddd-01-conn-nonblocking but uses a file descriptor passed directly by the application rather than BIO_s_connect.

Originally planned changes

• Change of method (as for ddd-01-conn-blocking);

• The arguments to the socket(2) call are changed from (AF_INET, SOCK_STREAM, IPPROTO_TCP) to (AF_INET, SOCK_DGRAM, IPPROTO_UDP);

• A change to how the POLLIN/POLLOUT/POLLERR flags to pass to poll(2) need to be determined.

• Additional functions in application code to determine event handling timeouts related to QUIC (get_conn_pump_timeout) and to pump the QUIC event loop (pump).

• Timeout computation code which involves merging and comparing different timeouts and calling pump as needed, based on deadlines reported by libssl.

Actual changes

The following additional changes needed to be made:

• Change of method name (as for ddd-01-conn-blocking);

• Use of ALPN (as for ddd-01-conn-blocking);

• SSL_get_timeout replaced with SSL_get_event_timeout (as for ddd-02-conn-nonblocking);

• SSL_pump renamed to SSL_handle_events (as for ddd-02-conn-nonblocking);

• The strategy for how to determine when to poll for POLLIN, when to poll for POLLOUT, etc. has changed since the original DDD process. This information is now exposed via SSL_net_read_desired and SSL_net_write_desired (as for ddd-02-conn-nonblocking).

ddd-05-mem-nonblocking

This demo is more elaborate. It uses memory buffers created and managed by an application as an intermediary between libssl and the network, which is a common usage pattern for applications. Managing this pattern for QUIC is more elaborate since datagram semantics on the network channel need to be maintained.

Originally planned changes

• Change of method (as for ddd-01-conn-blocking);

• Call to BIO_new_bio_pair is changed to BIO_new_dgram_pair, which provides a bidirectional memory buffer BIO with datagram semantics.

• A change to how the POLLIN/POLLOUT/POLLERR flags to pass to poll(2) need to be determined.

• Potential changes to buffer sizes used by applications to buffer datagrams, if those buffers are smaller than 1472 bytes.

• The arguments to the socket(2) call are changed from (AF_INET, SOCK_STREAM, IPPROTO_TCP) to (AF_INET, SOCK_DGRAM, IPPROTO_UDP);

Actual changes

The following additional changes needed to be made:

• Change of method name (as for ddd-01-conn-blocking);

• Use of ALPN (as for ddd-01-conn-blocking);

• The API to construct a BIO_s_dgram_pair ended up being named BIO_new_bio_dgram_pair rather than BIO_new_dgram_pair;

• Use of SSL_net_read_desired and SSL_net_write_desired (as for ddd-02-conn-nonblocking).

ddd-06-mem-uv

This demo is the most elaborate of the set. It uses a real-world asynchronous I/O reactor, namely libuv (the engine used by Node.js). In doing so it seeks to demonstrate and prove the viability of our API design with a real-world asynchronous I/O system. It operates wholly in non-blocking mode and uses memory buffers on either side of the QUIC stack to feed data to and from the application and the network.

Originally planned changes

• Change of method (as for ddd-01-conn-blocking);

• Various changes to use of libuv needed to switch to using UDP;

• Additional use of libuv to configure a timer event;

• Call to BIO_new_bio_pair is changed to BIO_new_dgram_pair (as for ddd-05-mem-nonblocking);

• Some reordering of code required by the design of libuv.

Actual changes

The following additional changes needed to be made:

• Change of method name (as for ddd-01-conn-blocking);

• Use of ALPN (as for ddd-01-conn-blocking);

• BIO_new_dgram_pair renamed to BIO_new_bio_dgram_pair (as for ddd-05-mem-nonblocking);

• SSL_get_timeout replaced with SSL_get_event_timeout (as for ddd-02-conn-nonblocking);

• SSL_pump renamed to SSL_handle_events (as for ddd-02-conn-nonblocking);

• Fixes to use of libuv based on a corrected understanding of its operation, and changes that necessarily ensue.

Conclusions

The DDD process has successfully delivered on the objective of delivering a QUIC API which can be used with only minimal API changes. The additional changes on top of those originally planned which were required to successfully execute the demos using QUIC were highly limited in scope and mostly constituted only minor changes. The sum total of the changes required for each demo (both planned and additional), as denoted in each DDD demo file under #ifdef USE_QUIC guards, are both minimal and limited in scope.

“Minimal” and “limited” are distinct criteria. If inexorable technical requirements dictate, an enormous set of changes to an application could be considered “minimal”. The changes required to representative applications, as demonstrated by the DDD demos, are not merely minimal but also limited.

For example, while the extent of these necessary changes varies by the sophistication of each demo and the kind of application usage pattern it represents, some demos in particular demonstrate exceptionally small changesets; for example, ddd-01-conn-blocking and ddd-02-conn-nonblocking-threads, with ddd-01-conn-blocking literally being enabled by a single line change assuming ALPN is already configured.

This report concludes the DDD process for the single-stream QUIC client API design process, which sought to validate our API design and API ease of use for existing applications seeking to adopt QUIC.