Skip to content
Snippets Groups Projects
  1. Apr 06, 2022
  2. Feb 07, 2022
  3. Jun 09, 2021
  4. Feb 02, 2021
    • Antoine Lambert's avatar
      Remove no longer used legacy Lister API and update CLI options · 89335445
      Antoine Lambert authored
      Legacy Lister classes from the swh.lister.core mdule are no longer
      used in swh-lister codebase so it is time to remove them.
      
      Also remove lister CLI options related to legacy Lister API.
      
      As a consequence, the following requirements are no longer needed:
      arrow, SQLAlchemy, sqlalchemy-stubs and testing.postgresql.
      
      Closes T2442
      89335445
  5. Jan 20, 2021
    • Tenma's avatar
      Reimplement Bitbucket lister using new Lister API · 565e7423
      Tenma authored
      The new lister has incremental and full listing capability.
      It can request the Bitbucket API in anonymous and HTTP basic authentication
      modes. Rate-limiting is not aggressive and is handled.
      565e7423
  6. Nov 23, 2020
  7. Nov 20, 2020
  8. Jul 09, 2020
  9. Oct 28, 2019
  10. Sep 03, 2019
    • David Douard's avatar
      implement listers as plugins · e3c0ea9d
      David Douard authored
      Listers are declared as plugins via the `swh.workers` entry_point.
      
      As such, the registry function is expected to return a dict with the
      `task_modules` field (as for generic worker plugins), plus:
      
      - `lister`: the lister class,
      - `models`: list of SQLAlchemy models used by this lister,
      - `init` (optionnal): hook (callable) used to initialize the lister's state
        (typically, create/initialize the database for this lister).
        If not set, the default implementation creates database tables (after
        optionally having deleted exisintg ones) according to models declared in
        the `models` register field.
      
      There is no need for explicitely add lister task modules in the main
      `conftest` module, but any new/extra lister to be tested must be registered
      (the tested lister module must be properly installed in the test environment).
      
      Also refactor a bit the cli tools:
      - add support for the standard --config-file option at the 'lister' group
        level,
      - move the --db-url to the 'lister' group,
      - drop the --lister option for the `swh lister db-init` cli tool:
        initializing (especially with --drop-tables) the database for a single
        lister is unreliable, since all tables are created using a sibgle MetaData
        (in the same namespace).
      e3c0ea9d
  11. Jan 16, 2019
  12. Oct 30, 2018
  13. Oct 30, 2017
Loading