Ingest Arch Linux

• swh/devel/swh-lister!423 (closed): Lister draft (not landed)
• swh/devel/swh-lister!423 (closed): Discussion about current caveats and possible improvements based on ^
• swh/devel/swh-lister!424 (closed), swh/devel/swh-lister!426 (closed), swh/devel/swh-lister!301 (closed): New lister implementation
• swh/devel/swh-lister!424 (closed), #4233: Lister run ok within docker
• swh/devel/swh-loader-core!433 (closed), swh/devel/swh-loader-core!435 (closed): Loader
• Loader run ok within docker
• swh/devel/swh-lister!426 (closed): Document Lister
• swh/devel/swh-loader-core!433 (closed): Document Loader
• swh/infra/sysadm-environment#5148 (closed): Deploy to staging
• Papercuts fixing
  • swh/devel/swh-loader-core#4746 (closed): Fails to retrieve most artifacts
• Call for public review on results ^
• Deploy in production when green light

---

Migrated from T4233 (view on Phabricator)

added Arch Lister Arch loader Archive coverage priority:Normal labels

changed the description

changed the description

changed the description

changed the description

marked the checklist item swh/devel/swh-lister!424 (closed): New lister implementation as completed

marked the checklist item swh/devel/swh-loader-core!433 (closed): Loader as completed

mentioned in commit swh/devel/swh-loader-core@01547b8e

mentioned in commit swh/devel/swh-lister@7dd412e5

Arch Linux Lister Docker Report

The lister takes a lot of time and fail on max retries when scraping repository directory (It has run fine a few weeks ago.). Not sure at this point, but I suspect that's a random problem related to network / http server. WIll run it multiple time to see if it failed on the same resource.

By the way I guess that the we need to define a strategy for those exceptions.

changed the description

changed the description

! In #4233, @franckbret wrote: Arch Linux Lister Docker Report

The lister takes a lot of time and fail on max retries when scraping repository directory (It has run fine a few weeks ago.). Not sure at this point, but I suspect that's a random problem related to network / http server. WIll run it multiple time to see if it failed on the same resource.

By the way I guess that the we need to define a strategy for those exceptions.

Looks like swh/devel/swh-lister!301 (closed) helps the lister complete successfully

Task swh.lister.arch.tasks.ArchListerTask[a27f6656-936b-40c1-a1de-23c626d9a7e3] succeeded in 2188.289039958996s: {'pages': 9, 'origins': 36638}

changed milestone to %Extend archive coverage [Roadmap - Collect]

removed Archive coverage label

added #4994 (closed) as child task

added #4995 as child task

added #4996 as child task

added #4997 as child task

added ll-status::in-progress label

marked the checklist item swh/devel/swh-lister!424 (closed), #4233: Lister run ok within docker as completed

added ll:ready-for-staging label and removed ll-status::in-progress label

mentioned in issue swh/infra/sysadm-environment#5148 (closed)

changed the description

added ll-status::staging label and removed ll:ready-for-staging label

marked the checklist item swh/infra/sysadm-environment#5148 (closed): Deploy to staging as completed

Deployed on staging but there is a blocking issue with it.

swh/devel/swh-loader-core#4746 (closed): Fails to retrieve most artifacts.

As added in the task description.

@bchauvet I've still moved this task in the board as deployed in staging. But I'd say this should be fixed prior to move further (that is prior to trigger the call for review).

added octobus-nlnet label

marked the checklist item Loader run ok within docker as completed