Skip to content

scheduler: Update configuration according to latest development

Antoine R. Dumont requested to merge generated-differential-D1117-source into generated-differential-D1117-target
  • listener: Migrate to use the swh-scheduler cli as entry point
  • runner: Migrate to use the swh-scheduler cli as entry point
  • scheduler-backend: Fix configuration key change
  • scheduler: Factorize configuration for listener/runner/api
  • Add missing celery configuration for listener/runner

Related #1525 (closed)

Test Plan

$ cd puppet-environment
$ bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to update_scheduler_config saatchi
Found host saatchi.internal.softwareheritage.org
Cloning into '/tmp/swh-ocd.LqmOZ9ED/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.LqmOZ9ED/environments/update_scheduler_config/data/private'...
done.
*** Running octocatalog-diff on host saatchi.internal.softwareheritage.org
I, [2019-02-13T16:21:49.260256 #11216]  INFO -- : Catalogs compiled for saatchi.internal.softwareheritage.org
I, [2019-02-13T16:21:50.026120 #11216]  INFO -- : Diffs computed for saatchi.internal.softwareheritage.org
diff origin/production/saatchi.internal.softwareheritage.org current/saatchi.internal.softwareheritage.org
*******************************************
  File[/etc/softwareheritage/backend/scheduler.yml] =>
   parameters =>
     content =>
      @@ -3,4 +3,4 @@
         cls: local
         args:
      -    scheduling_db: host=db dbname=softwareheritage-scheduler user=swhscheduler password=swh::deploy::scheduler::db::password
      +    db: host=db dbname=softwareheritage-scheduler user=swhscheduler password=swh::deploy::scheduler::db::password
      _
*******************************************
- File[/etc/softwareheritage/scheduler.ini]
*******************************************
+ File[/etc/softwareheritage/scheduler.yml] =>
   parameters =>
      "ensure": "present"
      "group": "swhscheduler"
      "mode": "0640"
      "notify": ["Service[swh-scheduler-listener]", "Service[swh-scheduler-runner]"]
      "owner": "root"
      "content": >>>
---
scheduler:
  cls: local
  args:
    db: host=db dbname=softwareheritage-scheduler user=swhscheduler password=swh::deploy::scheduler::db::password
celery:
  task_broker: amqp://swhproducer:swh::deploy::scheduler::task_broker::password@rabbitmq:5672//
<<<
*******************************************
  File[/etc/systemd/system/swh-scheduler-listener.service] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
       Group=swhscheduler
       Type=simple
      -ExecStart=/usr/bin/python3 -m swh.scheduler.celery_backend.listener
      +ExecStart=/usr/bin/swh-scheduler --config-file /etc/softwareheritage/scheduler.yml --log-level INFO listener
       Restart=always
       RestartSec=10
*******************************************
  File[/etc/systemd/system/swh-scheduler-runner.service] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
       Group=swhscheduler
       Type=simple
      -ExecStart=/bin/sh -c 'while true; do echo running pending tasks at `/bin/date`...; /usr/bin/python3 -m swh.scheduler.celery_backend.runner; sleep 10; done'
      +ExecStart=/usr/bin/swh-scheduler --config-file /etc/softwareheritage/scheduler.yml --log-level INFO runner --period 10
       Restart=always
       RestartSec=10
*******************************************
  Systemd::Unit_file[swh-scheduler-listener.service] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
       Group=swhscheduler
       Type=simple
      -ExecStart=/usr/bin/python3 -m swh.scheduler.celery_backend.listener
      +ExecStart=/usr/bin/swh-scheduler --config-file /etc/softwareheritage/scheduler.yml --log-level INFO listener
       Restart=always
       RestartSec=10
*******************************************
  Systemd::Unit_file[swh-scheduler-runner.service] =>
   parameters =>
     content =>
      @@ -12,5 +12,5 @@
       Group=swhscheduler
       Type=simple
      -ExecStart=/bin/sh -c 'while true; do echo running pending tasks at `/bin/date`...; /usr/bin/python3 -m swh.scheduler.celery_backend.runner; sleep 10; done'
      +ExecStart=/usr/bin/swh-scheduler --config-file /etc/softwareheritage/scheduler.yml --log-level INFO runner --period 10
       Restart=always
       RestartSec=10
*******************************************
*** End octocatalog-diff on saatchi.internal.softwareheritage.org

Migrated from D1117 (view on Phabricator)

Merge request reports