Add forge now - Process https://git.alarsyo.net

changed milestone to %Extend archive coverage [Roadmap - Collect]

changed the description

On staging environment:

swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/ \
>   add-forge-now --preset staging \
>     register-lister gitea \
>       url=https://git.alarsyo.net/api/v1/
Created 1 tasks

Task 33420957
  Next run: today (2023-04-05T15:31:57.302783+00:00)
  Interval: 90 days, 0:00:00
  Type: list-gitea-full
  Policy: oneshot
  Args:
  Keyword args:
    enable_origins: False
    max_origins_per_page: 10
    max_pages: 3
    url: 'https://git.alarsyo.net/api/v1/'

swhscheduler@scheduler0:~$ swh scheduler --url http://scheduler0.internal.staging.swh.network:5008/   add-forge-now --preset staging   schedule-first-visits     --type-name git     --lister-name gitea     --lister-instance-name git.alarsyo.net
98 slots available in celery queue
10 visits to send to celery

added 1h of time spent

On staging environment, all repositories from this forge was successfully visited:

swh-scheduler=> select visit_type, url, last_visit_status from origin_visit_stats
    where visit_type='git' and
    url like 'https://git.alarsyo.net%';
 visit_type |                            url                             | last_visit_status 
------------+------------------------------------------------------------+-------------------
 git        | https://git.alarsyo.net/alarsyo/advent-of-code.git         | successful
 git        | https://git.alarsyo.net/alarsyo/antoinemartin.fr.git       | successful
 git        | https://git.alarsyo.net/alarsyo/bad-news.git               | successful
 git        | https://git.alarsyo.net/alarsyo/beancount-cde-importer.git | successful
 git        | https://git.alarsyo.net/alarsyo/doom-conf.git              | successful
 git        | https://git.alarsyo.net/alarsyo/extract_har_urls.git       | successful
 git        | https://git.alarsyo.net/alarsyo/keymap.git                 | successful
 git        | https://git.alarsyo.net/alarsyo/lohr.git                   | successful
 git        | https://git.alarsyo.net/alarsyo/ltl2ba-nix.git             | successful
 git        | https://git.alarsyo.net/alarsyo/music.git                  | successful
(10 rows)

swh-scheduler=> select last_visit_status, count(ovs.url)
    from origin_visit_stats ovs
    join listed_origins lo USING(url, visit_type)
    where lister_id = (select id from listers
                       where name='gitea' and
                       instance_name='git.alarsyo.net')
    and visit_type='git'
    group by last_visit_status;
 last_visit_status | count 
-------------------+-------
 successful        |    10
(1 row)

So on production environment:

swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   add-forge-now --preset production \
>   register-lister gitea \
>     url=https://git.alarsyo.net/api/v1/
Created 1 tasks

Task 415365283
  Next run: today (2023-04-06T07:53:28.054613+00:00)
  Interval: 90 days, 0:00:00
  Type: list-gitea-full
  Policy: recurring
  Args:
  Keyword args:
    url: 'https://git.alarsyo.net/api/v1/'

swhscheduler@saatchi:~$ swh scheduler --url http://saatchi.internal.softwareheritage.org:5008/ \
>   add-forge-now --preset production \
>     schedule-first-visits \
>       --type-name git \
>       --lister-name gitea \
>       --lister-instance-name git.alarsyo.net
10000 slots available in celery queue
14 visits to send to celery

added 45m of time spent

The first ingestion is successfully completed.

softwareheritage-scheduler=> select visit_type, url, last_visit_status from origin_visit_stats
    where visit_type='git' and
    url like 'https://git.alarsyo.net%';
 visit_type |                            url                             | last_visit_status 
------------+------------------------------------------------------------+-------------------
 git        | https://git.alarsyo.net/alarsyo/antoinemartin.fr.git       | successful
 git        | https://git.alarsyo.net/alarsyo/bad-news.git               | successful
 git        | https://git.alarsyo.net/alarsyo/extract_har_urls.git       | successful
 git        | https://git.alarsyo.net/alarsyo/keymap.git                 | successful
 git        | https://git.alarsyo.net/alarsyo/ltl2ba-nix.git             | successful
 git        | https://git.alarsyo.net/alarsyo/music.git                  | successful
 git        | https://git.alarsyo.net/alarsyo/nixos-config.git           | successful
 git        | https://git.alarsyo.net/alarsyo/scratch-site.git           | successful
 git        | https://git.alarsyo.net/alarsyo/unicornlor.git             | successful
 git        | https://git.alarsyo.net/alarsyo/beancount-cde-importer.git | successful
 git        | https://git.alarsyo.net/alarsyo/doom-conf.git              | successful
 git        | https://git.alarsyo.net/alarsyo/lohr.git                   | successful
 git        | https://git.alarsyo.net/alarsyo/prololo-reborn.git         | successful
 git        | https://git.alarsyo.net/alarsyo/advent-of-code.git         | successful
(14 rows)

softwareheritage-scheduler=> select last_visit_status, count(ovs.url)
    from origin_visit_stats ovs
    join listed_origins lo USING(url, visit_type)
    where lister_id = (select id from listers
                       where name='gitea' and
                       instance_name='git.alarsyo.net')
    and visit_type='git'
    group by last_visit_status;
 last_visit_status | count 
-------------------+-------
 successful        |    14
(1 row)

added 30m of time spent

closed

Add forge now - Process https://git.alarsyo.net

Designs

Child items ...

Activity