cgit: handle "429 Too Many Requests" errors
For example:
HTTPError: 429 Client Error: Too Many Requests for url: https://git.alpinelinux.org/?ofs=10
File "swh/lister/cgit/tasks.py", line 20, in list_cgit
return lister.run().dict()
File "swh/lister/pattern.py", line 127, in run
for page in self.get_pages():
File "swh/lister/cgit/lister.py", line 90, in get_pages
bs_idx = self._get_and_parse(next_page)
File "swh/lister/cgit/lister.py", line 80, in _get_and_parse
response.raise_for_status()
File "requests/models.py", line 940, in raise_for_status
raise HTTPError(http_error_msg, response=self)
https://sentry.softwareheritage.org/share/issue/9316020c589242e587f9a71c607cb31a/
Migrated from T3645 (view on Phabricator)