Skip to content

List/Ingest major cgit instances

Number of origins reached the number of listed repositories, so ingestion is done.

|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|
| status | url                                         | url-prefix                        | instance       | # repos listed | # origins      |
|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|
| done   | https://git.kernel.org/                     | X                                 | git-kernel     |           1002 |           1002 |
| done   | https://gitweb.torproject.org/              | https://git.torproject.org/       | tor            |            492 |            501 |
| done   | https://fedorapeople.org/cgit/              | X                                 | fedora         |            751 |            841 |
| done   | https://cgit.kde.org/                       | https://anongit.kde.org/          | kde            |           2434 |           2434 |
| done   | https://www.happyassassin.net/cgit/         | X                                 | happyassassin  |              3 |              3 |
| done   | https://git.openembedded.org/               | X                                 | openembedded   |             15 |             15 |
| done   | https://git.zx2c4.com/                      | X                                 | zx2c4          |            140 |            140 |
| done   | http://git.gnu.org.ua/cgit/                 | http://git.gnu.org.ua/repo/       | git.gnu.org.ua |             50 |             50 |
| done   | https://git.alpinelinux.org/                | X                                 | alpinelinux    |            187 |            187 |
| done   | https://git.baserock.org/cgit/              | https://git.baserock.org/git/     | baserock       |           1456 |           1546 |
| done   | https://code.qt.io/cgit/                    | http://code.qt.io/                | qt.io          |            257 |            257 |
| done   | http://git.yoctoproject.org/clean/cgit.cgi/ | https://git.yoctoproject.org/git/ | yoctoproject   |            170 |            170 |
| done   | http://hdiff.luite.com/cgit/                | X                                 | hdiff.luite    |          13722 |          13722 |
|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|

(- https://anonscm.debian.org/cgit/: down as it got migrated to a gitlab instance, there seems to be an archive at https://alioth-archive.debian.org/)

  • Initialize db's data model with new table

  • Schedule the ready cgit instance listing [1]

with @nahimilega's support

  • [1]
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.savannah.gnu.org/cgit/ instance=gnu-savannah
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://gitweb.torproject.org/ url_prefix=https://git.torproject.org/ instance=tor
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.kernel.org/ instance=git-kernel
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://fedorapeople.org/cgit/ instance=fedora
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://cgit.kde.org/ url_prefix=https://anongit.kde.org/ instance=kde
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://www.happyassassin.net/cgit/ instance=happyassassin
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.openembedded.org/ instance=openembedded
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.zx2c4.com/ instance=zx2c4
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.gnu.org.ua/cgit/ url_prefix=http://git.gnu.org.ua/repo/ instance=git.gnu.org.ua
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://www.happyassassin.net/cgit/  instance=happyassassin
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.openembedded.org/ instance=openembedded
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.zx2c4.com/ instance=zx2c4
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.alpinelinux.org/ instance=alpinelinux
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://hdiff.luite.com/cgit/ instance=hdiff.luite
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.gnu.org.ua/cgit/                 url_prefix=http://git.gnu.org.ua/repo/       instance=git.gnu.org.ua
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.baserock.org/cgit/              url_prefix=https://git.baserock.org/git/     instance=baserock
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://code.qt.io/cgit/                    url_prefix=http://code.qt.io/                instance=qt.io
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.yoctoproject.org/clean/cgit.cgi/ url_prefix=https://git.yoctoproject.org/git/ instance=yoctoproject

Note:

  • The number of origins might be a tad superior to the number of listed repos because the count(*) on origin tables to compute the number might be not as accurate.
  • That means that the ingestion is either in progress or done by the way.

Migrated from T1835 (view on Phabricator)

Edited by Phabricator Migration user