Skip to content

Package up the common logic for the GitHub lister in a common class

The GitHub lister will serve as base for other listers. Some operations need to be factored out and cleaned up for other listers to be able to reuse/inspire themselves of.

  • Database schema management
  • à la swh.storage: one full SQL file with schema version + incremental updates
  • Initialization
  • Connection to the main database (origins)
  • Connection to the scheduler database (tasks)
  • Origin/Task creation (incremental and full operations)
  • Mass create tasks and get ids
  • Mass create origins and get ids
  • Task disabling (full operations)
  • Mass disable tasks from ids

We obviously need to make sure those operations exist in the underlying libraries (swh.storage, swh.scheduler)

Migrated from T588 (view on Phabricator)