Commits · 62dc4dc257fe13214d2f39d903033729ab847ac8 · Antoine R. Dumont / swh-lister

Nov 22, 2019
- Use pkg_resources to get the package version instead of vcversioner · 62dc4dc2
  Nicolas Dandrimont authored 5 years ago
  
  62dc4dc2
Nov 21, 2019
- lister.pypi: Align lister with pypi package loader · af04129d
  Antoine R. Dumont authored 5 years ago
  
  v0.0.43 Verified
  
  af04129d
- lister.npm: Align lister with npm package loader · 6534df41
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  6534df41
- lister.tests: Avoid duplication setup step · cb853f48
  Antoine R. Dumont authored 5 years ago
  
  And remove unnecessary fixture redefinition which causes indirection.
  Verified
  
  cb853f48
- Fix typos (and trailing ws) reported by codespell · 3ddfd00e
  David Douard authored 5 years ago
  
  3ddfd00e
- Add a pre-commit config file · dfd8569c
  David Douard authored 5 years ago
  
  dfd8569c
- cran/gnu: Rename task_type to load-archive-files · 1757b011
  Antoine R. Dumont authored 5 years ago
  
  v0.0.42 Verified
  
  1757b011
- lister.tests: Add missing task_type for package listers · 1cf7c8e8
  Antoine R. Dumont authored 5 years ago
  
  The scheduler module no longer initializes itself those task_type.
  Verified
  
  1cf7c8e8
- Migrate tox.ini to extras = xxx instead of deps = .[testing] · 5191d291
  Nicolas Dandrimont authored 5 years ago
  
  5191d291
- Merge tox environments · 0459a0bb
  Nicolas Dandrimont authored 5 years ago
  
  0459a0bb
Nov 20, 2019
- Include all requirements in MANIFEST.in · f9859f30
  Nicolas Dandrimont authored 5 years ago
  
  f9859f30
Nov 18, 2019
- lister.cli: Remove task type register cli · 484377cc
  Antoine R. Dumont authored 5 years ago
  
  It's now defined in swh.scheduler
  Verified
  
  484377cc
Nov 15, 2019
- simple_lister: Flush to db more frequently · 8d024586
  Antoine R. Dumont authored 5 years ago
  
  v0.0.41 Verified
  
  8d024586
- gnu.lister: Use url as primary key · bf030c0f
  Antoine R. Dumont authored 5 years ago
  
  Otherwise, we are failing unicity constraint. Related to T2070
  Verified
  
  bf030c0f
- gnu.lister.tests: Add missing assertion · daa9a270
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  daa9a270
- gnu.lister: Add missing retries_left parameter · 191043ff
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  191043ff
Nov 14, 2019
- debian.models: Migrate tests from storage to debian lister model · d2512012
  Antoine R. Dumont authored 5 years ago
  
  Related bb5d405
  Verified
  
  d2512012
Nov 13, 2019
- Fix bogus NotImplementedError on Area.index_uris · b2e5ce32
  Nicolas Dandrimont authored 5 years ago
  
  v0.0.40
  
  b2e5ce32
Nov 12, 2019
- indexing lister: Avoid generating empty or duplicate ranges when partitioning · 773cd337
  Nicolas Dandrimont authored 5 years ago
  
  v0.0.39
  
  773cd337
- indexing lister: Force bounds of integer ranges to be integers · 2c5528ef
  Nicolas Dandrimont authored 5 years ago
  
  2c5528ef
- Support zero-based indexes in indexing_lister · 1e7a905a
  Nicolas Dandrimont authored 5 years ago
  
  1e7a905a
- Add tests for IndexingLister.db_partition_indices function · e23960ed
  Nicolas Dandrimont authored 5 years ago
  
  e23960ed
- Register lister tasks even if they do not derive from SWHTask. · 8550c560
  vlorentz authored 5 years ago
  
  This happens if swh.scheduler.celery_backend.config isn't imported before the CLI runs.
  8550c560
Nov 08, 2019
- lister.debian: Actually use the db_engine passed to the hook function · ea7a08d0
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  ea7a08d0
- lister.cli: Clarify configuration loading step · eebbc859
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  eebbc859
Nov 06, 2019
- swh.lister: Remove completely references to swh.storage.schemata · e8a67a76
  Antoine R. Dumont authored 5 years ago
  
  Related to 56d7cff6
  v0.0.38 Verified
  
  e8a67a76
- tests: Bump dependency on latest swh-core · 81a31f3c
  Antoine R. Dumont authored 5 years ago
  
  This also modifies the test dataset to filename with url decoded filename. As this is what the latest pytest plugin requires.
  v0.0.37 Verified
  
  81a31f3c
Nov 05, 2019
- lister.debian: Move `run` method parameters to constructor · e0dbca75
  Antoine R. Dumont authored 5 years ago
  
  v0.0.36 Verified
  
  e0dbca75
- lister.debian: Default to run a listing on debian distribution · b745c5a7
  Antoine R. Dumont authored 5 years ago
  
  That fixes the `swh lister run --lister debian` cli entrypoint.
  Verified
  
  b745c5a7
- lister.debian: Fix task creation · a60e0bbc
  Antoine R. Dumont authored 5 years ago
  
  By adding a `retries_left`
  Verified
  
  a60e0bbc
- debian.lister: Send origin url as load-debian task parameter · f8727924
  Antoine R. Dumont authored 5 years ago
  
  Instead of the old origin dict. That's what the debian loaders (old and new) expect.
  Verified
  
  f8727924
Nov 04, 2019
- gnu.lister: Update docstrings and fix type annotations · 89b409d3
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  89b409d3
- gnu.lister: Unify timestamp formats to isoformat date in model · c6372eea
  Antoine R. Dumont authored 5 years ago
  
  Related T2023
  Verified
  
  c6372eea
- gnu.lister: Format timestamp to isoformat string for the tar loader · 93e7afda
  Antoine R. Dumont authored 5 years ago
  
  Related T2023
  Verified
  
  93e7afda
- gnu.tree: Move helper utilities function to the bottom of the file · 9fd64898
  Antoine R. Dumont authored 5 years ago
  
  Verified
  
  9fd64898
- lister.gnu: Move version parsing logic to the lister · 821f3f1c
  Antoine R. Dumont authored 5 years ago
  
  Related D2145 proposition
  Verified
  
  821f3f1c
- debian/lister: Use url parameter name instead of origin · 7c247c8a
  Antoine R. Dumont authored 5 years ago
  
  within the scheduled task. Related D2135
  Verified
  
  7c247c8a
- debian/model: Install lister model within the lister repository · 56d7cff6
  Antoine R. Dumont authored 5 years ago
  
  This is no longer shared between the new debian loader and the lister. The swh.storage.schemata module is still part of the swh.storage module though. As this is still a dependency for the current swh.loader.debian production loader. This will be cleaned up later. Related D2135
  Verified
  
  56d7cff6
Oct 28, 2019
- CRAN lister: fix compute_package_url interpolation · 44bc4462
  Stefano Zacchiroli authored 5 years ago
  
  44bc4462
- mypy: add typing annotations for novel lister abstractions · 6159faa2
  Stefano Zacchiroli authored 5 years ago
  
  6159faa2