List/Ingest major cgit instances
Number of origins reached the number of listed repositories, so ingestion is done.
|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|
| status | url | url-prefix | instance | # repos listed | # origins |
|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|
| done | https://git.kernel.org/ | X | git-kernel | 1002 | 1002 |
| done | https://gitweb.torproject.org/ | https://git.torproject.org/ | tor | 492 | 501 |
| done | https://fedorapeople.org/cgit/ | X | fedora | 751 | 841 |
| done | https://cgit.kde.org/ | https://anongit.kde.org/ | kde | 2434 | 2434 |
| done | https://www.happyassassin.net/cgit/ | X | happyassassin | 3 | 3 |
| done | https://git.openembedded.org/ | X | openembedded | 15 | 15 |
| done | https://git.zx2c4.com/ | X | zx2c4 | 140 | 140 |
| done | http://git.gnu.org.ua/cgit/ | http://git.gnu.org.ua/repo/ | git.gnu.org.ua | 50 | 50 |
| done | https://git.alpinelinux.org/ | X | alpinelinux | 187 | 187 |
| done | https://git.baserock.org/cgit/ | https://git.baserock.org/git/ | baserock | 1456 | 1546 |
| done | https://code.qt.io/cgit/ | http://code.qt.io/ | qt.io | 257 | 257 |
| done | http://git.yoctoproject.org/clean/cgit.cgi/ | https://git.yoctoproject.org/git/ | yoctoproject | 170 | 170 |
| done | http://hdiff.luite.com/cgit/ | X | hdiff.luite | 13722 | 13722 |
|--------+---------------------------------------------+-----------------------------------+----------------+----------------+----------------|
(- https://anonscm.debian.org/cgit/: down as it got migrated to a gitlab instance, there seems to be an archive at https://alioth-archive.debian.org/)
-
Initialize db's data model with new table -
Schedule the ready cgit instance listing [1]
with @nahimilega's support
- [1]
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.savannah.gnu.org/cgit/ instance=gnu-savannah
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://gitweb.torproject.org/ url_prefix=https://git.torproject.org/ instance=tor
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.kernel.org/ instance=git-kernel
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://fedorapeople.org/cgit/ instance=fedora
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://cgit.kde.org/ url_prefix=https://anongit.kde.org/ instance=kde
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://www.happyassassin.net/cgit/ instance=happyassassin
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.openembedded.org/ instance=openembedded
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.zx2c4.com/ instance=zx2c4
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.gnu.org.ua/cgit/ url_prefix=http://git.gnu.org.ua/repo/ instance=git.gnu.org.ua
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://www.happyassassin.net/cgit/ instance=happyassassin
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.openembedded.org/ instance=openembedded
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.zx2c4.com/ instance=zx2c4
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.alpinelinux.org/ instance=alpinelinux
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://hdiff.luite.com/cgit/ instance=hdiff.luite
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.gnu.org.ua/cgit/ url_prefix=http://git.gnu.org.ua/repo/ instance=git.gnu.org.ua
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://git.baserock.org/cgit/ url_prefix=https://git.baserock.org/git/ instance=baserock
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=https://code.qt.io/cgit/ url_prefix=http://code.qt.io/ instance=qt.io
SCHEDULER_API_URL=http://saatchi.internal.softwareheritage.org:5008/; swh scheduler --url $SCHEDULER_API_URL task add list-cgit url=http://git.yoctoproject.org/clean/cgit.cgi/ url_prefix=https://git.yoctoproject.org/git/ instance=yoctoproject
Note:
- The number of origins might be a tad superior to the number of listed repos because the
count(*)
on origin tables to compute the number might be not as accurate. - That means that the ingestion is either in progress or done by the way.
Migrated from T1835 (view on Phabricator)
Edited by Phabricator Migration user