Add forge now - process https://git.minetest.land

Gitea instance: https://archive.softwareheritage.org/admin/add-forge/request/19/

staging:

• Trigger add-forge-now listing [1]
• D8958: Fix swh.lister issue regarding gitea/gogs listing [1']
• Deploy fix (docker image) and trigger back listing
• Ensure listing happened [2]
• Schedule those origins for ingestion [3]
• Ensure ingestion is ongoing [4]

production:

• https://forge.softwareheritage.org/D8990: python3-swh.lister debian build stuck due to missing debian packages repomd [5]
• Trigger add-forge-now listing [6]
• Ensure listing happened
• Schedule those origins for ingestion [7]
• Checks [8]

[1]

swhscheduler@scheduler0:~/addforgenow$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ \
>   add-forge-now --preset staging \
>     register-lister gitea \
>       url=https://git.minetest.land/api/v1/
Created 1 tasks

Task 33420092
  Next run: today (2022-12-14T14:53:00.691631+00:00)
  Interval: 90 days, 0:00:00
  Type: list-gitea-full
  Policy: oneshot
  Args:
  Keyword args:
    enable_origins: False
    max_origins_per_page: 10
    max_pages: 3
    url: 'https://git.minetest.land/api/v1/'

[1']

listers TypeError: list_gitea_full() got an unexpected keyword argument 'max_pages'

[2]

│ listers [2022-12-14 15:42:37,882: INFO/ForkPoolWorker-2] Task swh.lister.gitea.tasks.FullGiteaRelister[90a9d6fa-a6ce-412f-bd17-759ad3e12cd4] succeeded in 1.9919601250439882s: {'pages': 3, 'origins': 30}

[3]

swhscheduler@scheduler0:~/addforgenow$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ \
>   add-forge-now --preset staging \
>   schedule-first-visits \
>     --type-name git \
>     --lister-name gitea \
>     --lister-instance-name git.minetest.land

100 slots available in celery queue
30 visits to send to celery

[4]

2022-12-14 16:47:13 swh-scheduler@db1:5432 λ select last_visit_status, count(ovs.url)
    from origin_visit_stats ovs
    join listed_origins lo USING(url, visit_type)
    where lister_id = (select id from listers
                       where name='gitea' and
                       instance_name='git.minetest.land')
    and visit_type='git'
    group by last_visit_status;
+-------------------+-------+
| last_visit_status | count |
+-------------------+-------+
| successful        |     4 |
| failed            |     1 |
| (null)            |    25 |
+-------------------+-------+
(3 rows)

Time: 1906.604 ms (00:01.907)

[5] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DLS/job/gbp-buildpackage/217/console

[6]

swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   add-forge-now \
>     register-lister gitea \
>       url=https://git.minetest.land/api/v1/
Created 1 tasks

Task 415264712
  Next run: today (2023-01-03T10:33:54.389254+00:00)
  Interval: 90 days, 0:00:00
  Type: list-gitea-full
  Policy: recurring
  Args:
  Keyword args:
    url: 'https://git.minetest.land/api/v1/'

Created 1 tasks

Task 415264713
  Next run: tomorrow (2023-01-04T10:33:54.470783+00:00)
  Interval: 1 day, 0:00:00
  Type: list-gitea-incremental
  Policy: recurring
  Args:
  Keyword args:
    url: 'https://git.minetest.land/api/v1/'

[7]

swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   add-forge-now \
>   schedule-first-visits \
>     --type-name git \
>     --lister-name gitea \
>     --lister-instance-name git.minetest.land
10000 slots available in celery queue
316 visits to send to celery

[8]

2023-01-03 11:40:01 softwareheritage-scheduler@belvedere:5432 λ select last_visit_status, count(ovs.url)
from origin_visit_stats ovs
join listed_origins lo USING(url, visit_type)
where lister_id = (select id from listers
                   where name='gitea' and
                   instance_name='git.minetest.land')
and visit_type='git'
group by last_visit_status;
+-------------------+-------+
| last_visit_status | count |
+-------------------+-------+
| successful        |    76 |
| (null)            |   240 |
+-------------------+-------+
(2 rows)

Time: 134229.820 ms (02:14.230)