Skip to content

Deploy Bzr loader to the staging environment

Now that the incremental loader has been committed, it seems like a good time to test it in the staging environment. While a more optimized version of the loader is still under development, the current version should be robust enough to handle anything archivable.

Plan:

  • D7117: swh-loader-bzr: Add module swh.loader.bzr.tasks module declared

  • Reference new swh-loader-bzr project in sentry [1]

  • Make loader run in docker

  • Prepare the necessary debian metadata files to allow CI package build

  • Debian packages built

    • unstable
    • D7132, D7133: stable is ko because conflit of breezy version (we need breezy > 3.1, only 3.0 is debian packaged).
    • backports python3-breezy and friends so the python3-swh.loader.bzr is finally happy on stable.
  • infra/puppet/puppet-swh-site!481: Prepare puppet manifest to deploy the worker service

  • D7124: Make the loader run in docker reproducible for dev

  • Actually deploy service on staging workers

  • Register the new load-bzr task-type in the scheduler (requires the python3-swh.loader.bzr install on the scheduler node)

  • Restart swh-scheduler-scheduler-recurrent service

  • Make it consume bzr tasks (D7172: sourceforge ones, launchpad ones?). T3945 to track remaining issues.

  • [1] https://sentry.softwareheritage.org/organizations/swh/issues/?project=22#welcome


Migrated from T3915 (view on Phabricator)

Edited by Antoine R. Dumont