lister.core: Remove origin creation from lister behavior
Closed
requested to merge generated-differential-D1579-source into generated-differential-D1579-target
Prior to this commit, lister did create origins as well in the archive (swh-storage's db). Now, we plan to only schedule those origins for ingestion (and we let the loaders do the actual creation).
Test Plan
tox
Then swh-docker-dev through multiple runs.
First run:
- checkout origin/master (without this diff)
- Add dummy/witness new task type loader-gnu (for the listing to actually succeed, implementation detail of the current lister/loader's status)
- Run gnu lister through scheduler
- Check swh-scheduler db (new tasks added)
- Check swh-storage db (new origins)
- Check swh-lister db (filled-in cache)
Second run:
- checkout this diff
- Add dummy/witness new task type loader-gnu
- Run gnu lister through scheduler
- Check swh-scheduler db (new tasks added)
- Check swh-storage db (no new origins as this behavior got removed here so \m/)
- Check swh-lister db (still filled-in cache)
Migrated from D1579 (view on Phabricator)
Merge request reports
Activity
Please register or sign in to reply