Skip to content
Snippets Groups Projects
Select Git revision
  • master default protected
  • swh_coar_notify
  • mr/fix-graph-declaration
  • mr/deploy-new-provenance
  • create_swh_coar
  • mr/list-dependencies
  • mr/bump-python3.11
  • mr/optimize-build-image-time
  • maven-exporter
  • mr/activate-statsd
  • improve-toolbox-msg
  • mr/memray-for-counters
  • wip/wait-for-pypi
  • drop-default-values
  • add-swh-utils-image
  • add-swh-toolbox-application
  • adapt-list-to-list-dependent-apps
  • swh-loader-git-pypy-20250404.1
  • swh-loader-savecodenow-20250404.1
  • swh-toolbox-20250404.1
  • swh-loader-svn-20250404.1
  • swh-datasets-20250404.2
  • swh-datasets-20250404.1
  • swh-coar-notify-20250404.1
  • swh-coar-notify-20250403.1
  • swh-provenance-20250403.2
  • swh-web-20250403.1
  • swh-provenance-20250403.1
  • swh-web-20250402.1
  • swh-provenance-20250402.2
  • swh-provenance-20250402.1
  • swh-loader-savecodenow-20250402.1
  • swh-toolbox-20250402.1
  • swh-loader-svn-20250402.1
  • swh-loader-package-20250402.1
  • swh-loader-metadata-20250402.1
  • swh-deposit-20250402.1
37 results
Blame Permalink
README.rst 7.06 KiB

Software Heritage virtual environment packaging manifests

This repository contains the manifests and scripts to automate the packaging of SWH applications into (fairly) reproducible Python virtual environments, and then onto container images.

Problem statement

To move away from our legacy Debian-based packaging and deployment workflow, we are using Python virtual environments to generate (fairly) reproducible deployment environments, that would be consistent between:

  • docker-based local environments
  • "bare VM/Metal" deployments (on Debian systems still managed by puppet)
  • elastic deployments based on k8s
  • CI environments used for testing packages

We want the input for generating these environments to be declarative (for instance, "I want an environment with swh.provenance"), and the resulting environments to be:

  • frozen (using a consistent, known set of package versions)
  • kept up to date automatically (for our swh packages, as well as the external dependencies)
  • tested before publication (at least to a minimal extent, e.g. ensuring that the tests of the declared input modules are successful before tagging an application as ready to deploy)

Packaging Workflow