Mathieu Velten 42dd992bb7 Dockerfile for tests: align comments with current behavior (#13867) | 2 years ago | |
---|---|---|
.. | ||
complement | 42dd992bb7 Dockerfile for tests: align comments with current behavior (#13867) | 2 years ago |
conf | 65e675504f Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152) | 2 years ago |
conf-workers | 166fafdf8d synapse-workers docker: copy nginx and redis in from base images (#13447) | 2 years ago |
Dockerfile | d5292b8017 Fix Docker build when Rust .so has been build locally first (#13811) | 2 years ago |
Dockerfile-dhvirtualenv | c9b7e97355 Add a stub Rust crate (#12595) | 2 years ago |
Dockerfile-workers | 166fafdf8d synapse-workers docker: copy nginx and redis in from base images (#13447) | 2 years ago |
README-testing.md | 10e4093839 Call out buildkit is required when building test docker images (#13338) | 2 years ago |
README.md | 2c5e2ae898 Document that the `DOCKER_BUILDKIT=1` flag is needed to build the docker image. (#13515) | 2 years ago |
build_debian.sh | d7141e0b8b Fix Shellcheck SC2006: Use $(...) notation | 3 years ago |
configure_workers_and_start.py | 65e675504f Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`. (#13152) | 2 years ago |
prefix-log | d743b25c8f Use supervisord to supervise Postgres and Caddy in the Complement image. (#12480) | 2 years ago |
start.py | 9667bad55d Improve startup times in Complement test runs against workers, particularly in CPU-constrained environments. (#13127) | 2 years ago |
It's possible to run integration tests against Synapse using Complement. Complement is a Matrix Spec compliance test suite for homeservers, and supports any homeserver docker image configured to listen on ports 8008/8448. This document contains instructions for building Synapse docker images that can be run inside Complement for testing purposes.
Note that running Synapse's unit tests from within the docker image is not supported.
scripts-dev/complement.sh
is a script that will automatically build
and run Synapse against Complement.
Consult the contributing guide for instructions on how to use it.
Under some circumstances, you may wish to build the images manually. The instructions below will lead you to doing that.
Note that these images can only be built using BuildKit,
therefore BuildKit needs to be enabled when calling docker build
. This can be done by
setting DOCKER_BUILDKIT=1
in your environment.
Start by building the base Synapse docker image. If you wish to run tests with the latest release of Synapse, instead of your current checkout, you can skip this step. From the root of the repository:
docker build -t matrixdotorg/synapse -f docker/Dockerfile .
Next, build the workerised Synapse docker image, which is a layer over the base image.
docker build -t matrixdotorg/synapse-workers -f docker/Dockerfile-workers .
Finally, build the multi-purpose image for Complement, which is a layer over the workers image.
docker build -t complement-synapse -f docker/complement/Dockerfile docker/complement
This will build an image with the tag complement-synapse
, which can be handed to
Complement for testing via the COMPLEMENT_BASE_IMAGE
environment variable. Refer to
Complement's documentation for
how to run the tests, as well as the various available command line flags.
See the Complement image README for information about the expected environment variables.
For manual testing of a multi-process Synapse instance in Docker, Dockerfile-workers is a Dockerfile that will produce an image bundling all necessary components together for a workerised homeserver instance.
This includes any desired Synapse worker processes, a nginx to route traffic accordingly, a redis for worker communication and a supervisord instance to start up and monitor all processes. You will need to provide your own postgres container to connect to, and TLS is not handled by the container.
Once you've built the image using the above instructions, you can run it. Be sure you've set up a volume according to the usual Synapse docker instructions. Then run something along the lines of:
docker run -d --name synapse \
--mount type=volume,src=synapse-data,dst=/data \
-p 8008:8008 \
-e SYNAPSE_SERVER_NAME=my.matrix.host \
-e SYNAPSE_REPORT_STATS=no \
-e POSTGRES_HOST=postgres \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=somesecret \
-e SYNAPSE_WORKER_TYPES=synchrotron,media_repository,user_dir \
-e SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK=1 \
matrixdotorg/synapse-workers
...substituting POSTGRES*
variables for those that match a postgres host you have
available (usually a running postgres docker container).
The SYNAPSE_WORKER_TYPES
environment variable is a comma-separated list of workers to
use when running the container. All possible worker names are defined by the keys of the
WORKERS_CONFIG
variable in this script, which the
Dockerfile makes use of to generate appropriate worker, nginx and supervisord config
files.
Sharding is supported for a subset of workers, in line with the
worker documentation. To run multiple instances of a given worker
type, simply specify the type multiple times in SYNAPSE_WORKER_TYPES
(e.g SYNAPSE_WORKER_TYPES=event_creator,event_creator...
).
Otherwise, SYNAPSE_WORKER_TYPES
can either be left empty or unset to spawn no workers
(leaving only the main process).
The container will only be configured to use Redis-based worker mode if there are
workers enabled.
Logs for workers and the main process are logged to stdout and can be viewed with
standard docker logs
tooling. Worker logs contain their worker name
after the timestamp.
Setting SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK=1
will cause worker logs to be written to
<data_dir>/logs/<worker_name>.log
. Logs are kept for 1 week and rotate every day at 00:
00, according to the container's clock. Logging for the main process must still be
configured by modifying the homeserver's log config in your Synapse data volume.
Setting the SYNAPSE_AS_REGISTRATION_DIR
environment variable to the path of
a directory (within the container) will cause the configuration script to scan
that directory for .yaml
/.yml
registration files.
Synapse will be configured to load these configuration files.
Nginx is present in the image to route requests to the appropriate workers, but it does not serve TLS by default.
You can configure SYNAPSE_TLS_CERT
and SYNAPSE_TLS_KEY
to point to a
TLS certificate and key (respectively), both in PEM (textual) format.
In this case, Nginx will additionally serve using HTTPS on port 8448.