Skip to content

swh-apps: Bump swh-* images to use python:3.11-bookworm

Antoine R. Dumont requested to merge mr/bump-python3.11 into master

This:

  • updates the docker images to bookworm
  • rework the Dockerfile scaffolding so we can share a base image between our apps [0] [1]
  • unifies again the entrypoint with more idiomatic shell command (regarding optional env variables)

From the review and pairing, some extra refactoring steps occurred too:

  • Drop all the "deeply inspired" comments ;-)
  • Reorder the content of the Dockerfiles to get better layer deduplication
  • Drop librdkafka-dev
  • Drop libcmph-dev (swh.perfecthash also bundles libcmph in its manylinux wheels)
  • Drop swh-loader-bzr bzr package (we'll see if that's still working without, at some point it required it to be functional)
  • Promote sed and git as default dependency in base image
  • Make loader-savecodenow image inherit from the loader-package
  • Update swh-toolbox dpes to use postgresql 15 client
  • Make swh-toolbox inherit from loader-savecodenow image (adapted from suggestion)

Note that this now requires buildkit to be enabled (so we can use smarter command like COPY --chmod=0644 ...)

Co-authored with @olasd and @vsellier

[0] There is currently a lot of duplication in the current declaration.

[1] This should make the maintenance of the Dockerfile easier. This also makes the overall build time faster. This is a work that will also open a more proper separation between docker build image and docker run image (to make the runtime images smaller, they are currently a bit fat, around ~1g of size, which in turn should reduce the waiting time in between deployments).

[2] sample build but all were tested time and again

$ app=loader-svn; DOCKER_BUILDKIT=1 docker build -t "swh-${app}:latest" "apps/swh-${app}" --build-arg REGISTRY=
[+] Building 0.1s (13/13) FINISHED
 => [internal] load build definition from Dockerfile                                                                                                                                                           0.0s
 => => transferring dockerfile: 964B                                                                                                                                                                           0.0s
 => [internal] load .dockerignore                                                                                                                                                                              0.0s
 => => transferring context: 2B                                                                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/swh-base:latest                                                                                                                                             0.0s
 => [internal] load metadata for docker.io/library/rsvndump-base:latest                                                                                                                                        0.0s
 => [rsvndump_image 1/1] FROM docker.io/library/rsvndump-base:latest                                                                                                                                           0.0s
 => [stage-1 1/6] FROM docker.io/library/swh-base:latest                                                                                                                                                       0.0s
 => [internal] load build context                                                                                                                                                                              0.0s
 => => transferring context: 78B                                                                                                                                                                               0.0s
 => CACHED [stage-1 2/6] COPY --from=rsvndump_image                                                                                                                                                            0.0s
 => CACHED [stage-1 3/6] RUN apt-get -y update &&     apt-get install -y subversion libsvn-dev &&     apt-get clean                                                                                            0.0s
 => CACHED [stage-1 4/6] COPY --chmod=0644 requirements-frozen.txt /opt/swh                                                                                                                                    0.0s
 => CACHED [stage-1 5/6] RUN --mount=type=cache,target=/opt/swh/.cache     uv pip sync requirements-frozen.txt                                                                                                 0.0s
 => CACHED [stage-1 6/6] COPY --chmod=0755 entrypoint.sh /opt/swh                                                                                                                                              0.0s
 => exporting to image                                                                                                                                                                                         0.0s
 => => exporting layers                                                                                                                                                                                        0.0s
 => => writing image sha256:be755312307ea94bf5db44a5d704997b789e60f479108707b7a1c360767b5616                                                                                                                   0.0s
 => => naming to docker.io/library/swh-loader-svn:latest                                                                                                                                                       0.0s

Refs. sysadm-environment#5560 (closed)

Edited by Antoine R. Dumont

Merge request reports

Loading