-
- Downloads
implement listers as plugins
Listers are declared as plugins via the `swh.workers` entry_point. As such, the registry function is expected to return a dict with the `task_modules` field (as for generic worker plugins), plus: - `lister`: the lister class, - `models`: list of SQLAlchemy models used by this lister, - `init` (optionnal): hook (callable) used to initialize the lister's state (typically, create/initialize the database for this lister). If not set, the default implementation creates database tables (after optionally having deleted exisintg ones) according to models declared in the `models` register field. There is no need for explicitely add lister task modules in the main `conftest` module, but any new/extra lister to be tested must be registered (the tested lister module must be properly installed in the test environment). Also refactor a bit the cli tools: - add support for the standard --config-file option at the 'lister' group level, - move the --db-url to the 'lister' group, - drop the --lister option for the `swh lister db-init` cli tool: initializing (especially with --drop-tables) the database for a single lister is unreliable, since all tables are created using a sibgle MetaData (in the same namespace).
Showing
- requirements-swh.txt 2 additions, 3 deletionsrequirements-swh.txt
- requirements-test.txt 1 addition, 1 deletionrequirements-test.txt
- setup.py 12 additions, 0 deletionssetup.py
- swh/lister/bitbucket/__init__.py 13 additions, 0 deletionsswh/lister/bitbucket/__init__.py
- swh/lister/cgit/__init__.py 13 additions, 0 deletionsswh/lister/cgit/__init__.py
- swh/lister/cli.py 65 additions, 160 deletionsswh/lister/cli.py
- swh/lister/core/models.py 23 additions, 0 deletionsswh/lister/core/models.py
- swh/lister/core/tests/conftest.py 0 additions, 18 deletionsswh/lister/core/tests/conftest.py
- swh/lister/cran/__init__.py 13 additions, 0 deletionsswh/lister/cran/__init__.py
- swh/lister/debian/__init__.py 40 additions, 0 deletionsswh/lister/debian/__init__.py
- swh/lister/github/__init__.py 13 additions, 0 deletionsswh/lister/github/__init__.py
- swh/lister/gitlab/__init__.py 13 additions, 0 deletionsswh/lister/gitlab/__init__.py
- swh/lister/gnu/__init__.py 13 additions, 0 deletionsswh/lister/gnu/__init__.py
- swh/lister/npm/__init__.py 13 additions, 0 deletionsswh/lister/npm/__init__.py
- swh/lister/packagist/__init__.py 13 additions, 0 deletionsswh/lister/packagist/__init__.py
- swh/lister/phabricator/__init__.py 13 additions, 0 deletionsswh/lister/phabricator/__init__.py
- swh/lister/pypi/__init__.py 13 additions, 0 deletionsswh/lister/pypi/__init__.py
- swh/lister/tests/test_cli.py 6 additions, 34 deletionsswh/lister/tests/test_cli.py
Loading
Please register or sign in to comment