Skip to content
Snippets Groups Projects

lister.core: Remove origin creation from lister behavior

Prior to this commit, lister did create origins as well in the archive (swh-storage's db). Now, we plan to only schedule those origins for ingestion (and we let the loaders do the actual creation).

Test Plan

tox

Then swh-docker-dev through multiple runs.

First run:

  • checkout origin/master (without this diff)
  • Add dummy/witness new task type loader-gnu (for the listing to actually succeed, implementation detail of the current lister/loader's status)
  • Run gnu lister through scheduler
  • Check swh-scheduler db (new tasks added)
  • Check swh-storage db (new origins)
  • Check swh-lister db (filled-in cache)

Second run:

  • checkout this diff
  • Add dummy/witness new task type loader-gnu
  • Run gnu lister through scheduler
  • Check swh-scheduler db (new tasks added)
  • Check swh-storage db (no new origins as this behavior got removed here so \m/)
  • Check swh-lister db (still filled-in cache)

Migrated from D1579 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading