Skip to content

launchpad: Ignore erratic page and continue listing

Antoine R. Dumont requested to merge generated-differential-D7198-source into generated-differential-D7198-target

In case of all retry tryouts failed, finally ignore that page and continue listing the next pages.

Related to #3948 (closed)

Test Plan

tox

Applied on staging and triggered the listing. That actually allowed the listing to finish properly.

Feb 17 17:40:37 worker1 python3[2168786]: [2022-02-17 17:40:37,571: INFO/MainProcess] Received task: swh.lister.gitlab.tasks.IncrementalGitLabLister[496b82a5-6e7a-4e39-841c-e9158334fc1e]
Feb 17 17:40:37 worker1 python3[2168796]: [2022-02-17 17:40:37,796: INFO/ForkPoolWorker-4] Task swh.lister.gitlab.tasks.IncrementalGitLabLister[496b82a5-6e7a-4e39-841c-e9158334fc1e] succeeded in 0.19229740649461746s: {'pages': 1, 'origins': 0}

It finished:

Feb 17 20:36:17 worker1 python3[2168793]: [2022-02-17 20:36:17,861: INFO/ForkPoolWorker-1] Task swh.lister.launchpad.tasks.FullLaunchpadLister[d8bb1c71-4c2c-406c-9431-f358cdca5ec5] succeeded in 10058.709554322995s: {'pages': 2, 'origins':
216789}

Migrated from D7198 (view on Phabricator)

Merge request reports