Skip to content

Ensure the loader/lister packages are installed on scheduler node

Antoine R. Dumont requested to merge generated-differential-D7803-source into generated-differential-D7803-target

This allows to trigger the task type registration cli properly without having to install the missing package manually first.

Encountered the issue when deploying the bzr loader. task-type registration cli would not do anything without installing the package first. Which is expected given how we rely the setuptools entrypoint for it.

Related to T4214

Test Plan

octo-diff on scheduler instances wants to install those missing packages deps:

$SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging scheduler0.internal.staging.swh.network
Found host scheduler0.internal.staging.swh.network
WARN     -> Environment "staging-add-prometheus-metrics" contained non-word characters, correcting name to staging_add_prometheus_metrics
WARN     -> Environment "staging-bullseye-rabbitmq-plugin" contained non-word characters, correcting name to staging_bullseye_rabbitmq_plugin
WARN     -> Environment "staging-check-journal-client" contained non-word characters, correcting name to staging_check_journal_client
WARN     -> Environment "staging-check-journal-client-2nd-implementation" contained non-word characters, correcting name to staging_check_journal_client_2nd_implementation
WARN     -> Environment "staging-check-journal-client-first-implem" contained non-word characters, correcting name to staging_check_journal_client_first_implem
WARN     -> Environment "staging-pin" contained non-word characters, correcting name to staging_pin
Cloning into '/tmp/swh-ocd.aQjsyLpC/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.aQjsyLpC/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host scheduler0.internal.staging.swh.network
I, [2022-05-10T15:10:04.642395 #1615927]  INFO -- : Catalogs compiled for scheduler0.internal.staging.swh.network
I, [2022-05-10T15:10:04.999984 #1615927]  INFO -- : Diffs computed for scheduler0.internal.staging.swh.network
diff origin/production/scheduler0.internal.staging.swh.network current/scheduler0.internal.staging.swh.network
*******************************************
+ Package[python3-swh.lister] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.bzr] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.core] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.git] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.mercurial] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.svn] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
*** End octocatalog-diff on scheduler0.internal.staging.swh.network
 swh  tony  yavin4  ~  work  …  swh  puppet-environment  swh-site   production  10⚑  ERROR  %  $SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to staging saatchi
Found host saatchi.internal.softwareheritage.org
WARN     -> Environment "staging-add-prometheus-metrics" contained non-word characters, correcting name to staging_add_prometheus_metrics
WARN     -> Environment "staging-bullseye-rabbitmq-plugin" contained non-word characters, correcting name to staging_bullseye_rabbitmq_plugin
WARN     -> Environment "staging-check-journal-client" contained non-word characters, correcting name to staging_check_journal_client
WARN     -> Environment "staging-check-journal-client-2nd-implementation" contained non-word characters, correcting name to staging_check_journal_client_2nd_implementation
WARN     -> Environment "staging-check-journal-client-first-implem" contained non-word characters, correcting name to staging_check_journal_client_first_implem
WARN     -> Environment "staging-pin" contained non-word characters, correcting name to staging_pin
Cloning into '/tmp/swh-ocd.ovKO306D/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.ovKO306D/environments/staging/data/private'...
done.
*** Running octocatalog-diff on host saatchi.internal.softwareheritage.org
I, [2022-05-10T15:10:18.473634 #1618648]  INFO -- : Catalogs compiled for saatchi.internal.softwareheritage.org
I, [2022-05-10T15:10:18.840683 #1618648]  INFO -- : Diffs computed for saatchi.internal.softwareheritage.org
diff origin/production/saatchi.internal.softwareheritage.org current/saatchi.internal.softwareheritage.org
*******************************************
+ Package[python3-swh.lister] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.bzr] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.core] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.git] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.mercurial] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
+ Package[python3-swh.loader.svn] =>
   parameters =>
      "ensure": "installed"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner-priority]", "Service[swh-scheduler-runner]", "Service[swh-scheduler-schedule-recurrent]"]
*******************************************
*** End octocatalog-diff on saatchi.internal.softwareheritage.org

Migrated from D7803 (view on Phabricator)

Merge request reports