Production-readiness improvements for the GitHub lister (!213) · Merge requests · Platform / Development / swh-lister · GitLab

Snippets Groups Projects

Closed Nicolas Dandrimont requested to merge generated-differential-D5152-source into generated-differential-D5152-target 4 years ago

Mar 19, 2021
- GitHub: handle edge cases with empty responses · 879170a5
  Nicolas Dandrimont authored 4 years ago
  
  879170a5
- GitHub: handle Server Errors · c375a61b
  Nicolas Dandrimont authored 4 years ago
  
  These errors happen, sometimes, when requesting large pages of results.
  c375a61b
- GitHub: Move rate-limit reset logic to RateLimited exception · 4a215e68
  Nicolas Dandrimont authored 4 years ago
  
  This makes the logic easier to test.
  4a215e68
- Retry GitHub requests on ChunkEncodingErrors · cfd4169b
  Nicolas Dandrimont authored 4 years ago
  
  These happen, sometimes, when the connection to the GitHub server resets, e.g. because of congestion on a slow link.
  cfd4169b
- GitHub: Move rate limit handling to the request function · 61c1d444
  Nicolas Dandrimont authored 4 years ago
  
  61c1d444
- GitHub: Start moving the request logic to a separate function · 03b10e5c
  Nicolas Dandrimont authored 4 years ago
  
  03b10e5c
- GitHub: Use function for requests.Session initialization · 8f7dbb74
  Nicolas Dandrimont authored 4 years ago
  
  This will help us to break the retry logic for the listing requests themselves to a separate function too.
  8f7dbb74