Bottom line is that there is indeed no suitable API on the Sourcehut side and that web crawling would not be ideal as it is a privacy feature that without knowing usernames one cannot find their repositories.
In the future they are planning to add a suitable GraphQL API (presumably taking into account desired user privacy settings), but it does not exist yet.
We should put this on hold until that API exists (or develop it ourselves).