[Add Forge Now] process https://git.gnupg.org/cgi-bin/gitweb.cgi
Production request: https://archive.softwareheritage.org/admin/add-forge/request/255/
Type: gitweb
As this forge need an url parameter to register the lister, it will never pass the pipeline.
Activity
changed milestone to %Extend archive coverage [Roadmap - Collect]
added AddForgeNow label
assigned to @guillaume
On each repository page there are two URL, one with
httpsscheme, the other withgitscheme.
Only thegitone can be cloned:/tmp ᐅ git clone https://git.gnupg.org/libksba.git Cloning into 'libksba'... fatal: repository 'https://git.gnupg.org/libksba.git/' not found /tmp ᐅ git clone git://git.gnupg.org/libksba.git Cloning into 'libksba'... remote: Enumerating objects: 3880, done. remote: Counting objects: 100% (3880/3880), done. remote: Compressing objects: 100% (3760/3760), done. remote: Total 3880 (delta 3020), reused 118 (delta 106), pack-reused 0 Receiving objects: 100% (3880/3880), 1.50 MiB | 4.67 MiB/s, done. Resolving deltas: 100% (3020/3020), done.When I made some tests on
swh-environment, thebase_git_urlseems to be ignored:swh@d37ce3fe00ab:/$ swh scheduler -C $SWH_CONFIG_FILENAME \ add-forge-now --preset staging \ register-lister gitweb \ url=https://git.gnupg.org/cgi-bin/gitweb.cgi \ base_git_url=git://git.gnupg.org Created 1 tasks Task 2 Next run: today (2023-12-12T14:59:47.617858+00:00) Interval: 1 day, 0:00:00 Type: list-gitweb Policy: oneshot Args: Keyword args: base_git_url: 'git://git.gnupg.org' enable_origins: False max_origins_per_page: 10 max_pages: 3 url: 'https://git.gnupg.org/cgi-bin/gitweb.cgi' swh@d37ce3fe00ab:/$ swh scheduler -C $SWH_CONFIG_FILENAME origin check-listed-origins gitweb git.gnupg.org Forge git.gnupg.org (gitweb) has 10 listed origins in the scheduler database. swh@d37ce3fe00ab:/$ swh scheduler -C $SWH_CONFIG_FILENAME origin check-listed-origins gitweb git.gnupg.org -l url last_seen last_update -------------------------------------- -------------------------------- -------------------------------- https://git.gnupg.org/gnupg-doc.git 2023-12-12 14:59:55.770478+00:00 2023-12-01 14:59:54.267205+00:00 https://git.gnupg.org/gnupg.git 2023-12-12 14:59:55.770478+00:00 2023-12-12 05:59:54.428068+00:00 https://git.gnupg.org/gpa.git 2023-12-12 14:59:55.770478+00:00 2023-08-12 14:59:55.766838+00:00 https://git.gnupg.org/gpgme.git 2023-12-12 14:59:55.770478+00:00 2023-11-30 14:59:54.608611+00:00 https://git.gnupg.org/libassuan.git 2023-12-12 14:59:55.770478+00:00 2023-10-31 14:59:54.789170+00:00 https://git.gnupg.org/libgcrypt.git 2023-12-12 14:59:55.770478+00:00 2023-12-12 06:59:54.942813+00:00 https://git.gnupg.org/libgpg-error.git 2023-12-12 14:59:55.770478+00:00 2023-12-11 08:59:55.120524+00:00 https://git.gnupg.org/libksba.git 2023-12-12 14:59:55.770478+00:00 2023-11-21 14:59:55.267090+00:00 https://git.gnupg.org/npth.git 2023-12-12 14:59:55.770478+00:00 2023-09-12 14:59:55.435354+00:00 https://git.gnupg.org/ntbtls.git 2023-12-12 14:59:55.770478+00:00 2023-09-12 14:59:55.596347+00:00 Forge git.gnupg.org (gitweb) has 10 listed origins in the scheduler database. swh@d37ce3fe00ab:/$ swh scheduler -C $SWH_CONFIG_FILENAME \ add-forge-now --preset staging \ schedule-first-visits \ --type-name git \ --lister-name gitweb \ --lister-instance-name git.gnupg.org 1000 slots available in celery queue 10 visits to send to celery swh@d37ce3fe00ab:/$ swh scheduler -C $SWH_CONFIG_FILENAME origin check-ingested-origins gitweb git.gnupg.org -l url last_visit_status last_visit -------------------------------------- ------------------- -------------------------------- https://git.gnupg.org/gnupg-doc.git not_found 2023-12-12 15:01:32.247637+00:00 https://git.gnupg.org/gnupg.git not_found 2023-12-12 15:01:32.440954+00:00 https://git.gnupg.org/gpgme.git not_found 2023-12-12 15:01:35.842499+00:00 https://git.gnupg.org/gpa.git not_found 2023-12-12 15:01:35.649892+00:00 https://git.gnupg.org/libassuan.git not_found 2023-12-12 15:01:36.032276+00:00 https://git.gnupg.org/npth.git not_found 2023-12-12 15:01:36.761230+00:00 https://git.gnupg.org/ntbtls.git not_found 2023-12-12 15:01:36.957773+00:00 https://git.gnupg.org/libksba.git not_found 2023-12-12 15:01:36.582400+00:00 https://git.gnupg.org/libgpg-error.git not_found 2023-12-12 15:01:36.391789+00:00 https://git.gnupg.org/libgcrypt.git not_found 2023-12-12 15:01:36.215035+00:00 Forge git.gnupg.org (gitweb) has 10 scheduled ingests in the scheduler. failed : 0 None : 0 not_found : 10 successful : 0 total : 10 success rate : 0.0%unassigned @guillaume
mentioned in issue swh/devel/swh-lister#4713
Please register or sign in to reply