Ingest gotweb from Game Of Trees
Summary:
A lister is needed for gotweb, a simple web frontend for git repositories.
gotweb instances can be listed by downloading and parsing the front page, then downloading and parsing each of the project pages.
The front page has links to repos and their ages in format 2 days ago
, so a naive lister looks like this:
curl -sL "https://got.gameoftrees.org/" |
pup 'div.index_wrapper div.index_project a attr{href}' |
sed "s_^_https://got.gameoftrees.org/_;s/&/\&/"
The clone URLs can be extracted from repo pages like this:
curl -sL 'https://got.gameoftrees.org/?action=summary&path=diff.git' |
pup 'html body main dl#summary_wrapper dd pre.clone-url text{}'
gotweb instances sometimes require the use of anonymous SSH to clone repositories, which I assume isn't a supported git clone
transport for SWH yet AFAIK?
git clone ssh://anonymous@got.gameoftrees.org/got.git
There are several instances, currently most are just mirrors but one has the OpenBSD code in it too:
https://got.gameoftrees.org/
https://got.bsdbox.org/
https://got.omarpolo.com/
https://got.jamsek.dev/
https://jamsek.net/
https://got.sexy.is/
https://gothub.org/
https://got.x61.sh/
Plan:
-
Implement lister -
Implement loader -
docker: Run lister -
docker: Run loader -
Document lister -
Document loader -
Deploy on staging -
Call for public review -
Deploy on production
Edited by Paul Wise