Explicit support of multiple Python versions for CI of swh Python modules in Jenkins
We currently have multiple development and deployment workflows that use separate Python versions:
- Elastic deployments in staging and production are based on containers that (currently) use Python 3.10
Legacy production deployments use Debian packages deployed on Buster, which supports Python 3.7- the docker development environment is based on the
python:3.7python:3.11 docker image - our jenkins-based CI uses a
Debian buster base image, which runs Python 3.7multipython image with support for every version between 3.7 and 3.12 but we have Debian package builds for Debian unstable, which run tests of released modules against Python 3.11- most developers will soon (if not already) be running Debian Bookworm, which will ship with Python 3.11
In practice, our main dev and CI workflows use Python 3.10 (matching the production deployments), and we don't have support for using a different Python version in these tests, so we don't get automated feedback on breakage introduced by new Python versions. Python3.7 support is also over, and a lot of upstreams have gotten rid of it. Finally, while we don't make support claims to third-party users, there's nothing preventing them from running our code in different python versions. All in all, we need to do the work to upgrade our current CI to a newer Python version, hopefully in a way that allows future upgrades more easily.
Our workflows use tox, which makes the use of different python versions really easy (just tox run py3xx
to have tox find a python3.xx interpreter and run tests against it). Jenkins currently uses the default python3 interpreter to run the module tests (it runs tests on the "py3" environment).
Here's a breakdown of the subtasks that this entails:
-
Create docker images for our CI environment with multiple versions of Python -
Add support in our Jenkins pipelines to run tests on a list of Python interpreter versions -
Define what pipeline runs should run tests on which Python versions -
Update jenkins pipeline definitions to implement the decision above -
Document our Python support matrix preeminently