Ingest humungus
Summary:
A lister is needed for humungus, a web frontend for Mercurial/hg repositories.
Every page has quotes from the 1981 movie The Road Warrior
, for example all pages have i am gravely disappointed
in the header, then front pages have i await your answer
and repo pages have i understand your pain
.
The front page has a p
with content public repos
and then after it the repo URLs are in a table
, one per tr
, in the first td
, in the a
href
attribute.
All the repo pages have URLs starting with /r/
and can be used with hg clone
.
A naive shell lister looks like this:
curl -sL $site | pup "a attr{href}" | sort -u | sed -n "s_^/r/_$site/r/_p"
According to Google, there is only one site:
https://humungus.tedunangst.com/
Plan:
-
Implement lister -
Implement loader -
docker: Run lister -
docker: Run loader -
Document lister -
Document loader -
Deploy on staging -
Call for public review -
Deploy on production