github: fill-in the gap

New dataset to schedule.

priority.list.github

Refs. #4847 (closed)

Refs. #4872 (closed)

Scheduled:

$ export SWH_CONFIG_FILENAME=~/.config/swh/scheduler.yml; cat ~/downloads/20230509-1539-priority.list.github | shuf | ./schedule_with_queue_length_check.py --queue-name oneshot-large-git
{'args': [], 'kwargs': {'url': 'https://github.com/liyuanhe211/Energy_Diagram_Plotter_CDXML', 'lister_name': 'github', 'lister_instance_name': 'github', 'pack_size_bytes': 34359738368}}
{'args': [], 'kwargs': {'url': 'https://github.com/wjm41/molplotly', 'lister_name': 'github', 'lister_instance_name': 'github', 'pack_size_bytes': 34359738368}}
{'args': [], 'kwargs': {'url': 'https://github.com/inbo/INBOmd', 'lister_name': 'github', 'lister_instance_name': 'github', 'pack_size_bytes': 34359738368}}
{'args': [], 'kwargs': {'url': 'https://github.com/QuantumPackage/qp2', 'lister_name': 'github', 'lister_instance_name': 'github', 'pack_size_bytes': 34359738368}}
{'args': [], 'kwargs': {'url': 'https://github.com/Genometric/MSPC', 'lister_name': 'github', 'lister_instance_name': 'github', 'pack_size_bytes': 34359738368}}
...

changed milestone to %Extend archive coverage [Roadmap - Collect]

added activity::Processing label

assigned to @ardumont

mentioned in commit swh/infra/ci-cd/swh-charts@866627bd

It was a tad long to ingest because it was stuck behind other large origins to ingest [1]

Anyway, remains 5 messages in the queue. So almost done.

[1] #4847 (closed)

added 1h of time spent

Randomly checking some origins [1] and all their visits are around may 2023, the 22/23 [2].

All good, closing.

[1]

$ shuf ~/downloads/20230509-1539-priority.list.github | head -1
https://github.com/ATFutures/dodgr
$ shuf ~/downloads/20230509-1539-priority.list.github | head -1
https://github.com/CPS-TUWien/racing_dreamer
$ shuf ~/downloads/20230509-1539-priority.list.github | head -1
https://github.com/vision-agh/pointpillars-backbone
...

[2] See

https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/Zzh-tju/Rotated-LD
https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/ATFutures/dodgr
https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/CPS-TUWien/racing_dreamer
https://archive.softwareheritage.org/browse/origin/visits/?origin_url=https://github.com/vision-agh/pointpillars-backbone

added 15m of time spent

closed