-
- Downloads
Refactor and deduplicate HTTP requests code in listers
Numerous listers were using the same page_request method or equivalent in their implementation so prefer to deduplicate that code by adding an http_request method in base lister class: swh.lister.pattern.Lister. That method simply wraps a call to requests.Session.request and logs some useful info for debugging and error reporting, also an HTTPError will be raised if a request ends up with an error. All listers using that new method now benefit of requests retry when an HTTP error occurs thanks to the use of the http_retry decorator.
Showing
- docs/new_lister_template.py 3 additions, 38 deletionsdocs/new_lister_template.py
- swh/lister/__init__.py 4 additions, 3 deletionsswh/lister/__init__.py
- swh/lister/arch/lister.py 2 additions, 29 deletionsswh/lister/arch/lister.py
- swh/lister/aur/lister.py 1 addition, 3 deletionsswh/lister/aur/lister.py
- swh/lister/bitbucket/lister.py 4 additions, 29 deletionsswh/lister/bitbucket/lister.py
- swh/lister/bitbucket/tests/test_lister.py 6 additions, 4 deletionsswh/lister/bitbucket/tests/test_lister.py
- swh/lister/bower/lister.py 4 additions, 31 deletionsswh/lister/bower/lister.py
- swh/lister/cgit/lister.py 3 additions, 12 deletionsswh/lister/cgit/lister.py
- swh/lister/cgit/tests/test_lister.py 2 additions, 2 deletionsswh/lister/cgit/tests/test_lister.py
- swh/lister/debian/lister.py 7 additions, 10 deletionsswh/lister/debian/lister.py
- swh/lister/gitlab/lister.py 2 additions, 7 deletionsswh/lister/gitlab/lister.py
- swh/lister/gogs/lister.py 19 additions, 33 deletionsswh/lister/gogs/lister.py
- swh/lister/gogs/tests/test_lister.py 3 additions, 3 deletionsswh/lister/gogs/tests/test_lister.py
- swh/lister/golang/lister.py 2 additions, 25 deletionsswh/lister/golang/lister.py
- swh/lister/golang/tests/test_lister.py 5 additions, 4 deletionsswh/lister/golang/tests/test_lister.py
- swh/lister/maven/lister.py 15 additions, 37 deletionsswh/lister/maven/lister.py
- swh/lister/maven/tests/test_lister.py 1 addition, 1 deletionswh/lister/maven/tests/test_lister.py
- swh/lister/npm/lister.py 5 additions, 25 deletionsswh/lister/npm/lister.py
- swh/lister/npm/tests/test_lister.py 14 additions, 8 deletionsswh/lister/npm/tests/test_lister.py
- swh/lister/packagist/lister.py 4 additions, 21 deletionsswh/lister/packagist/lister.py
Loading
Please register or sign in to comment