Ingest hgweb
Summary:
A lister is needed for hgweb, a web frontend for Mercurial/hg repositories.
hgweb sites have Mercurial repositories index
in the page title.
The repository URLs are in the table.bigtable tr td a
elements, in the href
attribute. On some large sites like the Mozilla one, subpages need to be loaded to discover all of the repositories.
According to Google and Bing it is used on sites below. Some are archival sites, some of them do have broken TLS, but all of them have repositories.
http://122.155.17.110:8080/
http://160.99.2.25:8080/
http://185.231.5.201:8080/
http://204.8.75.187:8080/
http://37.205.9.135/
http://hg.blasux.ru/
http://hg.devl.cz/
http://hg.eldalin.com/
http://hg.honeyplanet.jp/
http://hg.kewl.org/pub/
http://hg.localdomain.org/
http://hg.loongnix.cn/
http://hg.muempf.de/
http://hg.opensecurity.at/repos
http://hg.slions.net/
http://hg.strabon.di.uoa.gr/
http://hg.tryton.org/
http://k0s.org/hg
http://nuclear.mutantstargoat.com/hg/
http://oss.ucis.nl/hg/
https://anonhg.netbsd.org/
https://basil.dsg.cs.tcd.ie/code/n4c
https://borgac.net/lbo/hg/
https://cavac.at/public/mercurial
https://code.heinrichsweikamp.com/public
https://code.matthewwild.co.uk/
https://code.nsnam.org/
https://code.nsnam.org/
https://code.openbravo.com/erp
https://code.openbravo.com/erp/
https://code.openbravo.com/pos
https://code.openbravo.com/tools
https://code.unixwork.de/hg/
https://currents.soest.hawaii.edu/hgstage/
https://dev.ibboard.co.uk/repos/
https://dvcs.csnsm.in2p3.fr/DAQ-Public
https://dvcs.w3.org/hg/
https://gremmielabs.dev/
https://hg.256bit.org/
https://hg.alliedmods.net/
https://hg.alliedmods.net/hl2sdks
https://hg.beremiz.org/
https://hg.bgreco.net/
https://hg.cerbelle.net/hg
https://hg.defun.work/
https://hg.die-offenbachs.homelinux.org/
https://hg.die-offenbachs.homelinux.org/ericplugins/
https://hg.dwimlabs.net/
https://hg.edgewall.org/
https://hg.eldalin.com/
https://hg.five-ten-sg.com/
https://hg.fossee.in/
https://hg.frantovo.cz/
https://hg.glitchvid.com/hg/
https://hg.globalcode.info/
https://hg.gnu.org.ua/hgweb
https://hg.gregor-horvath.com/
https://hg.grin.hu/
https://hg.guido-berhoerster.org/
https://hg.gunet.gr/
https://hg.hedgewars.org/
https://hg.implab.org/
https://hg.jetbrains.org/hg/
https://hg.kakakhel.io/
https://hg.linkmauve.fr/
https://hg.localdomain.org/
https://hg.lunch.org.uk/
https://hg.malikania.fr/
https://hg.mbse.eu/
https://hg.mozilla.org/
https://hg.mrzv.org/
https://hg.nanaya.net/
https://hg.nimmagadda.net/
https://hg.octave.org/
https://hg.openindiana.org/
https://hg.pfish.zone/
https://hg.publiforge.org/
https://hgpub.locaixo.com/
https://hg.python.org/
https://hg.reportlab.com/hg-public/
https://hg.risc.jku.at/
https://hg.schlittermann.de/hg
https://hg.sdfa3.org/
https://hg.ucc.asn.au/
https://hg.uni-konstanz.de/
https://hg.videolan.org/
https://hg.ztfy.org/
https://hg.osdn.net/view/
https://icedtea.wildebeest.org/hg/
https://icedtea.wildebeest.org/people/
https://jan.vrany.io/hg
https://kryshen.net/hg/
https://mdounin.ru/hg/
https://nuclear.mutantstargoat.com/hg/
http://source.apidesign.org/hg/
https://pmeerw.net/hg
https://repo.mercurial-scm.org/
https://repos.goffi.org/
https://src.nth.io/
https://tnsp.org/hg/batmud/
https://toolshed.g2.bx.psu.edu/repos/
https://traz.dinauz.org/hg/
https://tt.note1.51share.fun:3154/
https://tuomov.iki.fi/repos/
https://wim.vree.org/wsgi/hgweb.wsgi/
https://www.akawolf.org/oss/
https://www.eighty-twenty.org/hgwebdir.cgi
https://www.freecalypso.org/hg/
https://www.ilamb.org/hg_repos/benchmark_releases
https://www.landley.net/hg/
https://www.ldeo.columbia.edu/cgi-bin/ladcp-cgi-bin/hgweb.cgi
https://www.linuxtv.org/hg
https://www.linuxtv.org/hg/
https://www.monetdb.org/hg
https://www.pauloang.com/hg/
https://www.public-software-group.org/mercurial/
https://www.retrodev.com/repos/
https://www.tnsp.org/hg/
https://www.yatex.org/hgrepos/
http://voa3r.dev.acta.asso.fr/
http://www.bortreb.com/hgweb.py
http://www.cr.ie.u-ryukyu.ac.jp/hg
http://www.eighty-twenty.org/hgwebdir.cgi
http://www.freediameter.net/hg
Plan:
-
Implement lister -
Implement loader -
docker: Run lister -
docker: Run loader -
Document lister -
Document loader -
Deploy on staging -
Call for public review -
Deploy on production
Edited by Paul Wise