Codeberg archivals are getting rate-limited

For a few months, our archival processes have been regularly rate limited by codeberg.org. This is particularly noticeable (and noticed) as Guix has started migrating some of their repository hosting to Codeberg.

This rate limiting was recently put in place to mitigate the onslaught of LLM scrapers.

There are some ways where SWH needs to improve visit scheduling before we ask Codeberg for a rate limiting uplift. For instance, swh/devel/swh-scheduler#4691 (closed) is definitely a defect on our side that generates server load and should be fixed.

Once we've reduced useless queries on our end, we should review whether a rate limit increase should be requested, and if so, create a ticket upstream at https://codeberg.org/Codeberg-e.V./requests/issues