Add a swh scheduler origin send-to-celery subcommand

The subcommand bypasses the legacy task-based mechanism to directly send new origin visits to celery

Test Plan

simple command line test added to check the basic behavior

Depends on !251 (closed)

---

Migrated from D6146 (view on Phabricator)

Build is green

Patch application report for D6146 (id=22245)

Could not rebase; Attempt merge onto cc76a573...

Updating cc76a57..2efad28
Fast-forward
 swh/scheduler/backend.py               |  9 ++++++-
 swh/scheduler/cli/origin.py            | 46 ++++++++++++++++++++++++++++++++++
 swh/scheduler/interface.py             |  3 +++
 swh/scheduler/tests/test_cli_origin.py | 34 +++++++++++++++++++++++++
 swh/scheduler/tests/test_scheduler.py  | 16 ++++++++++++
 5 files changed, 107 insertions(+), 1 deletion(-)

Changes applied before test

commit 2efad289833e971594833b9ed825b9acead8d254
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jun 1 20:04:11 2021 +0200

    Add a swh scheduler origin send-to-celery subcommand
    
    The subcommand bypasses the legacy task-based mechanism to directly send
    new origin visits to celery

commit 5e8007fdbfeb612ea394f97eeba25a1c4e529b7e
Author: Nicolas Dandrimont <nicolas@dandrimont.eu>
Date:   Tue Jun 1 15:48:05 2021 +0200

    Add table sampling option to grab_next_visits
    
    Running common operations on all git origins is pretty intense. Using
    table sampling gives us the opportunity to at least schedule some jobs
    in (decently small) time.

See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/447/ for more details.

mentioned in merge request !234 (closed)

Merge request was accepted

approved this merge request

Merge request was merged

closed