Commits · 89b409d30f9533456473cb4d948514f28fe4fa96 · Antoine R. Dumont / swh-lister

Nov 04, 2019
- gnu.lister: Update docstrings and fix type annotations · 89b409d3
  Antoine R. Dumont authored 5 years ago
  
  89b409d3
- gnu.lister: Unify timestamp formats to isoformat date in model · c6372eea
  Antoine R. Dumont authored 5 years ago
  
  Related T2023
  c6372eea
- gnu.lister: Format timestamp to isoformat string for the tar loader · 93e7afda
  Antoine R. Dumont authored 5 years ago
  
  Related T2023
  93e7afda
- gnu.tree: Move helper utilities function to the bottom of the file · 9fd64898
  Antoine R. Dumont authored 5 years ago
  
  9fd64898
- lister.gnu: Move version parsing logic to the lister · 821f3f1c
  Antoine R. Dumont authored 5 years ago
  
  Related D2145 proposition
  821f3f1c
- debian/lister: Use url parameter name instead of origin · 7c247c8a
  Antoine R. Dumont authored 5 years ago
  
  within the scheduled task. Related D2135
  7c247c8a
- debian/model: Install lister model within the lister repository · 56d7cff6
  Antoine R. Dumont authored 5 years ago
  
  This is no longer shared between the new debian loader and the lister. The swh.storage.schemata module is still part of the swh.storage module though. As this is still a dependency for the current swh.loader.debian production loader. This will be cleaned up later. Related D2135
  56d7cff6
Oct 28, 2019
- CRAN lister: fix compute_package_url interpolation · 44bc4462
  Stefano Zacchiroli authored 5 years ago
  
  44bc4462
- mypy: add typing annotations for novel lister abstractions · 6159faa2
  Stefano Zacchiroli authored 5 years ago
  
  6159faa2
- CRAN lister: make shelling out decoding compatible with Python 3.5 · 7dfd811e
  Stefano Zacchiroli authored 5 years ago
  
  7dfd811e
- typing: minimal changes to make a no-op mypy run pass · 974f80f9
  Stefano Zacchiroli authored 5 years ago
  
  974f80f9
Oct 18, 2019

Stop binding tasks to a specific instance of the celery app · 78105940

The celery.shared_task decorator allows late-binding of tasks to any celery app,
which is well suited for our "task plugin" architecture.

78105940

Oct 15, 2019
- debian.lister: Add integration test which checks scheduled tasks · a64ae964
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  a64ae964
- pypi.tests: Remove trailing _ in test method name · 960868ba
  Antoine R. Dumont authored 5 years ago
  
  960868ba
- npm.lister: Align docstrings · b73c657e
  Antoine R. Dumont authored 5 years ago
  
  b73c657e
- npm.tests: Add an integration test on listing with pagination · b4867ccd
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  b4867ccd
Oct 14, 2019
- tests: Update pytest_plugin according to latest version change · a8cde12d
  Antoine R. Dumont authored 5 years ago
  
  a8cde12d
- tox.ini: Install ipdb dependency for development environment · 9a1314c6
  Antoine R. Dumont authored 5 years ago
  
  9a1314c6
- tests/conftest: Log the db url used by tests · fcd85216
  Antoine R. Dumont authored 5 years ago
  
  fcd85216
Oct 12, 2019
- lister.core.db_utils: Remove dead code · 6b1c3d1f
  Antoine R. Dumont authored 5 years ago
  
  6b1c3d1f
- bitbucket.lister: Add integration test which checks scheduled tasks · f92ac836
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  f92ac836
- github.lister: Add integration test which checks scheduled tasks · 0b8b1419
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  0b8b1419
- gitlab.lister: Add integration test which checks scheduled tasks · 1889875f
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  1889875f
- tox.ini: Install a py3-dev environment to ease development · 2a33f430
  Antoine R. Dumont authored 5 years ago
  
  Failing tests is less noisy in development mode.
  2a33f430
Oct 11, 2019
- phabricator.lister: Add integration test which checks scheduled tasks · 903b644c
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  903b644c
- packagist.lister: Add integration test which checks scheduled tasks · f3bf9ae5
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  f3bf9ae5
- npm.lister: Add integration test which checks the scheduled tasks · 678b7ea5
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  678b7ea5
- pypi.lister: Add integration test which checks the scheduled tasks · 599af25a
  Antoine R. Dumont authored 5 years ago
  
  Related T2032
  599af25a
- cran.lister: Fix cran lister and add proper integration test · 8d50e0d9
  Antoine R. Dumont authored 5 years ago
  
  Which checks the cran lister tasks written in the scheduler. Related d30d574d Related 5ea9d5ed Related T2032
  8d50e0d9
Oct 10, 2019
- gnu.lister: Move tests datadir into its dedicated folder · ef2c1847
  Antoine R. Dumont authored 5 years ago
  
  Relatd D2076#inline-13551
  ef2c1847
Oct 09, 2019
- gnu.tests: Checks lister output from scheduler · 0f0b8401
  Antoine R. Dumont authored 5 years ago
  
  This also adds an swh-listers fixture which allows to retrieve a test ready lister from its name (e.g gnu). Those listers have access to a scheduler fixture so we can check the listing output from the scheduler instance.
  0f0b8401
- cgit.tests: Check the tasks from the scheduler · 394658e5
  Antoine R. Dumont authored 5 years ago
  
  394658e5
- simple_lister: Extract common behavior in base class · 04ca3186
  Antoine R. Dumont authored 5 years ago
  
  04ca3186
- core.models: Fix typo · 61ce38a0
  Antoine R. Dumont authored 5 years ago
  
  61ce38a0
- tox.ini: Use tests installed files instead of working directory · 4f30cb50
  Antoine R. Dumont authored 5 years ago
  
  Related D2082
  4f30cb50
Oct 04, 2019
- lister/gnu: Modify gnu lister's loading task creation · 3ce6c5c6
  Antoine R. Dumont authored 5 years ago
  
  Loader Task signature for the loader gnu is now: - args: - package - package urls - kwargs: tarballs: List of Dict with keys archive (unchanged), 'time' (was 'date'), length (new)
  3ce6c5c6
- lister/gnu: Remove unneeded get_file method · 00bb6c7b
  Antoine R. Dumont authored 5 years ago
  
  00bb6c7b
- lister/gnu: Modify default policy to oneshot · 322c9dc7
  Antoine R. Dumont authored 5 years ago
  
  322c9dc7
Oct 02, 2019

cran.lister: Refactor and fix cran lister · d30d574d

Antoine R. Dumont authored 5 years ago

Prior to this commit, the code was actually duplicated with an old version
which would not work.

Related D1492#41287

d30d574d

Sep 20, 2019

setup.py: Kill deprecated swh-lister command · 85d00106

Antoine R. Dumont authored 5 years ago

Prior to this commit, the pip activation environment failed because the old cli
name no longer exists, it's named 'lister' now.

85d00106