swh-apps: Bump swh-* images to use python:3.11-bookworm
This:
- updates the docker images to bookworm
- rework the Dockerfile scaffolding so we can share a base image between our apps [0] [1]
- unifies again the entrypoint with more idiomatic shell command (regarding optional env variables)
From the review and pairing, some extra refactoring steps occurred too:
-
Drop all the "deeply inspired" comments ;-) -
Reorder the content of the Dockerfiles to get better layer deduplication -
Drop librdkafka-dev -
Drop libcmph-dev (swh.perfecthash also bundles libcmph in its manylinux wheels) -
Drop swh-loader-bzr bzr package (we'll see if that's still working without, at some point it required it to be functional) -
Promote sed and git as default dependency in base image -
Make loader-savecodenow image inherit from the loader-package -
Update swh-toolbox dpes to use postgresql 15 client -
Make swh-toolbox inherit from loader-savecodenow image (adapted from suggestion)
Note that this now requires buildkit to be enabled (so we can use smarter command like COPY --chmod=0644 ...
)
Co-authored with @olasd and @vsellier
[0] There is currently a lot of duplication in the current declaration.
[1] This should make the maintenance of the Dockerfile easier. This also makes the overall build time faster. This is a work that will also open a more proper separation between docker build image and docker run image (to make the runtime images smaller, they are currently a bit fat, around ~1g of size, which in turn should reduce the waiting time in between deployments).
[2] sample build but all were tested time and again
$ app=loader-svn; DOCKER_BUILDKIT=1 docker build -t "swh-${app}:latest" "apps/swh-${app}" --build-arg REGISTRY=
[+] Building 0.1s (13/13) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 964B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/swh-base:latest 0.0s
=> [internal] load metadata for docker.io/library/rsvndump-base:latest 0.0s
=> [rsvndump_image 1/1] FROM docker.io/library/rsvndump-base:latest 0.0s
=> [stage-1 1/6] FROM docker.io/library/swh-base:latest 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 78B 0.0s
=> CACHED [stage-1 2/6] COPY --from=rsvndump_image 0.0s
=> CACHED [stage-1 3/6] RUN apt-get -y update && apt-get install -y subversion libsvn-dev && apt-get clean 0.0s
=> CACHED [stage-1 4/6] COPY --chmod=0644 requirements-frozen.txt /opt/swh 0.0s
=> CACHED [stage-1 5/6] RUN --mount=type=cache,target=/opt/swh/.cache uv pip sync requirements-frozen.txt 0.0s
=> CACHED [stage-1 6/6] COPY --chmod=0755 entrypoint.sh /opt/swh 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:be755312307ea94bf5db44a5d704997b789e60f479108707b7a1c360767b5616 0.0s
=> => naming to docker.io/library/swh-loader-svn:latest 0.0s
Edited by Antoine R. Dumont