Skip to content

loading some svn origins while ignoring history sometimes raises

Recreated from #3695 submitted by @ardumont as the associated diffs do not fix the original issue (my bad, i misread the task details).

It should allow to start the svn ingestion while ignoring previous ingestion result. It current raises while it should not.

The following stacktrack (KeyError) [1] shows some assertions were wrong. So this needs investigation and fix.

  • [1]
swhworker@worker17:~$ swh loader run svn https://profs.scienze.univr.it/posenato/svn/sw/CSTNU start_from_scratch=True
INFO:swh.loader.svn.SvnLoader:Load origin 'https://profs.scienze.univr.it/posenato/svn/sw/CSTNU' with type 'svn'
INFO:swh.loader.svn.SvnLoader:Processing revisions [1-619] for {'swh-origin': 'https://profs.scienze.univr.it/posenato/svn/sw/CSTNU', 'remote_url': 'https://profs.scienze.univr.it/posenato/svn/sw/CSTNU', 'local_url': b'/tmp/swh.loader.svn.qhb2pmup-99331/CSTNU', 'uuid': b'782453a1-1937-45d1-8845-2a6fcc2839b7'}
ERROR:swh.loader.svn.SvnLoader:90
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 457, in fetch_data
    data = next(self.swh_revision_gen)
  File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 364, in process_svn_revisions
    rev, commit, dir_id, revision_parents[rev]
KeyError: 90
ERROR:swh.loader.svn.SvnLoader:Loading failure, updating to `failed` status
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/swh/loader/core/loader.py", line 339, in load
    self.store_data()
  File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 489, in store_data
    revision=self._last_revision, snapshot=self._snapshot
  File "/usr/lib/python3/dist-packages/swh/loader/svn/loader.py", line 525, in generate_and_load_snapshot
    "generate_and_load_snapshot called with null revision and snapshot!"
ValueError: generate_and_load_snapshot called with null revision and snapshot!
{'status': 'failed'}

Note: start-from-scratch parameter should probably be renamed ignore-history along the way. That'd unify with the equivalent parameter in the loader-git.

Migrated from T3701 (view on Phabricator)