Skip to content

Make the http backend retry queries if configured for

Add support for an "retry" configuration entry that sets up and configure an urllib3 Retry HTTPAdapter.

Notes:

  • no idea how to properly test this; requests_mock does hijack the session and bypasses the adapters, so it does not "simply work" here.
  • a reason for adding this is to be able to set a retry policy for the s3 objstorage of a content replayer so it cat get a better chance for the object to be uploaded there by the main swh->s3 replayer when the consumer (a mirror) is processing this content... (aka reduce the number of fallback to azure when a mirror replayer is in steady state mode)
Edited by David Douard

Merge request reports