Skip to content
Snippets Groups Projects

docker: Explicit the scheduler runner for save-code-now

Closed docker: Explicit the scheduler runner for save-code-now
generated-differential-D5851-source into generated-differential-D5851-target
1 unresolved thread
Closed Antoine R. Dumont requested to merge generated-differential-D5851-source into generated-differential-D5851-target
1 unresolved thread

Related to swh/infra/sysadm-environment#3367 (closed)

Depends on swh-scheduler!236 (closed) (and the override in the docker-compose.override.yml for swh-scheduler-runner and swh-scheduler-runner-priority)

Test Plan

doco up -d Then play with the save code now and ensure the priority tasks are indeed prioritize with the right new scheduler (it is):

swh-scheduler-runner-priority_1  | wait-for-it: amqp:5672 is available after 0 seconds
swh-scheduler-runner-priority_1  | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks load-git (priority)
swh-scheduler-runner-priority_1  | INFO:swh.scheduler.cli.admin.runner:Scheduled 1 tasks

The other ones don't bulge unless you start listing stuff:

swh-scheduler-runner_1           |  postgresql is up!
swh-scheduler-runner_1           | (1 row)
swh-scheduler-runner_1           |
swh-scheduler-runner_1           | wait-for-it: waiting for swh-scheduler:5008 without a timeout
swh-scheduler-runner_1           | wait-for-it: swh-scheduler:5008 is available after 0 seconds
swh-scheduler-runner_1           | Starting the swh-scheduler start-runner
swh-scheduler-runner_1           | wait-for-it: waiting for amqp:5672 without a timeout
swh-scheduler-runner_1           | wait-for-it: amqp:5672 is available after 0 seconds
swh-scheduler-runner_1           | INFO:swh.scheduler.celery_backend.runner:Grabbed 1 tasks index-origin-metadata
swh-scheduler-runner_1           | INFO:swh.scheduler.cli.admin.runner:Scheduled 1 tasks
...

Migrated from D5851 (view on Phabricator)

Merge request reports

Approved by
Vincent Sellier

Closed by Phabricator Migration userPhabricator Migration user 3 years ago (Jun 10, 2021 2:01pm UTC)

Merge details

  • The changes were not merged into generated-differential-D5851-target.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
docker/docker-compose.yml
144 volumes:
145 - "./conf/scheduler.yml:/scheduler.yml:ro"
146 - "./services/swh-scheduler-worker/entrypoint.sh:/entrypoint.sh:ro"
147
148 swh-scheduler-runner-priority:
149 image: swh/stack
150 build: ./
151 env_file:
152 - ./env/common_python.env
153 - ./env/scheduler-db.env
154 - ./env/scheduler.env
155 environment:
156 SWH_CONFIG_FILENAME: /scheduler.yml
157 SWH_SCHEDULER_CONFIG_FILE: /scheduler.yml
158 entrypoint: /entrypoint.sh
159 command: start-runner --period 10 --with-priority --task-type load-git --task-type load-svn --task-type load-hg --task-type load-archive-files
  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    The difference is here, the runner is in charge of scheduling solely the load-{git,svn,hg,archive-files} with priority. The other (initial) runner won't do it anymore, it will only scheduled the rests of the tasks without any priority.

  • Please register or sign in to reply
  • Vincent Sellier
    Vincent Sellier @vsellier ·
    Maintainer

    LGTM

  • Vincent Sellier
    Vincent Sellier @vsellier ·
    Maintainer

    Merge request was accepted

  • Vincent Sellier approved this merge request

    approved this merge request

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    Merge request was merged

  • Antoine R. Dumont closed

    closed

    • Please register or sign in to reply