Skip to content

Prepare the tests to run in Jenkins

Jérémy Bobbio (Lunar) requested to merge generated-differential-D8634-source into generated-differential-D8634-target
  • Add missing requirement needed by the tests
  • Add build system to pyproject
  • Match SERVICES with current stack description
  • Stop waiting for monitoring services
  • Restart graph replayer automatically
  • Control Docker more robustly
  • Move API URL to an environment variable
  • Allow connections to swh-web from anywhere

To focus on the most important set of changes:

Previously, Docker was controlled through custom shell commands issued via pytest-testinfra. Multiple loops where used to wait for containers to converge to the desired state. Given the test requires quite some resources in terms of RAM, CPU, and bandwith, its execution can take between 20 minutes and a full hour. Therefore, it would be better to avoid having to edit the code to fiddle with timeouts and loop counts.

One of the way we used to wait for completion was looking at the Docker service logs for specific messages. Sadly, pytest-testinfra does not really provide a way to monitor the output of a command as it runs.

All-in-all, it felt easier to replace the custom shell commands by a library offering a Python API to control Docker. While dockerpy is the official library, its future maintainance by Docker has been in flux (see: https://github.com/docker/docker-py/pull/2989) and more importantly, as it speaks directly to the Docker socket, it does not support anything like docker stack.

Meanwhile, the python_on_whales has been featured on the official Docker blog (https://www.docker.com/blog/guest-post-calling-the-docker-cli-from-python-with-python-on-whales/) and implements as much as possible of Docker command line interface (by shelling out).

With it, it was possible to:

  • Replace all command lines with Python calls.
  • Wait for containers to shut down on teardown before removing volumes.
  • Wait for log messages to appear by blocking on the service output.
  • Wait for Docker readyness when we start the stack and scale services.

Some refactoring and naming improvements have been made along the way.

Because we still would like to timeout in case of a problem, pytest-timeout has been added to the requirements with a default timeout of 30 minutes.

Migrated from D8634 (view on Phabricator)

Merge request reports