- Jun 03, 2022
-
-
Antoine R. Dumont authored
Related to T4305
-
vlorentz authored
Needed since package loaders now create release objects instead of revision objects.
-
vlorentz authored
This will make it easier to support indexing from releases in the future, as it will remove the strong dependency on revision ids in the database and interfaces. The existence of the indexer/table is mostly to deduplicate work between origins with the same head revision, and we do not use it outside this context, so this should have no impact. The DB migration works by dropping both tables and re-indexing from scratch; which is necessary as we need to replace revision ids with directory ids.
-
- Jun 02, 2022
-
-
vlorentz authored
'local' is a deprecated alias
-
- Jun 01, 2022
- May 31, 2022
-
-
David Douard authored
instead of (soon-to-be-deprecated) swh-core's postgresql_fact one.
-
vlorentz authored
We stopped using it as an indexer years ago, so it does not make sense to keep this class around. Additionally, replace the weird dict (needed by the indexer interface) with a CoreSWHID (now possible, as it doesn't need to be JSON-like)
-
vlorentz authored
A future commit will significantly change test initialization, and using fixtures simplifies this.
-
- May 25, 2022
-
-
vlorentz authored
Before this commit, the journal client only created scheduler tasks, which then run the indexers. This commit adds support for a new flow: skipping the scheduler, to run indexers directly. This new behavior is triggered by adding a new argument on the CLI, which is the name of the indexer to run (currently, only `origin-intrinsic-metadata`). This has the following consequences: * a crash in an indexer will now hang the whole thing (which is arguably good) * the journal client will probably need to be parallelized to keep up with the load * we can remove an existence check for origins In term of deployment: 1. stop the old journal client 2. wait for all tasks to finish 3. stop and remove celery workers and queues 4. start the new journal client (it can reuse the group_id to avoid re-indexing, but I think it is a good opportunity to reindex because of all the temporary failures we had over time)
-
vlorentz authored
It will be used in a future commit to run indexers directly from a journal client.
-
vlorentz authored
A future commit will need pytest parametrization
-
- May 24, 2022
-
-
vlorentz authored
It may have unexpected side-effects
-
- May 12, 2022
- May 09, 2022
-
-
Pratyush authored
-
- May 02, 2022
-
-
Benoit Chauvet authored
-
- Apr 26, 2022
-
-
vlorentz authored
-
- Apr 21, 2022
-
-
Antoine Lambert authored
That hook can be frustrating as it can discard a long commit message if it finds a typo in it so better removing it.
-
Satvik authored
-
- Apr 19, 2022
-
-
Satvik authored
-
- Apr 08, 2022
-
-
Antoine Lambert authored
-
Antoine Lambert authored
Related to T3922
-
Antoine Lambert authored
black is considered stable since release 22.1.0 and the version we are currently using is quite outdated and not compatible with click 8.1.0, so it is time to bump it to its latest stable release. Please note that E501 pycodestyle warning related to line length is replaced by B950 one from flake8-bugbear as recommended by black. https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length Related to T3922
-
- Apr 06, 2022
-
-
Antoine Lambert authored
pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for pytest >= 7 so we can now drop the pytest pinning.
-
- Mar 22, 2022
-
-
Antoine Lambert authored
Due to test modules being copied in subdirectories of the build directory by setuptools, it makes pytest fail by raising ImportPathMismatchError exceptions when invoked from root directory of the module. So ignore the build folder to discover tests.
-
- Feb 24, 2022
-
-
David Douard authored
- add the `get_datastore` function in `swh.indexer` - add the `get_current_version` method in `IndexerBackend`, - remove dbversion management from sql init script - update tests accordingly
-
vlorentz authored
-
- Feb 10, 2022
-
-
Antoine Lambert authored
To install the new hook: $ pre-commit install -t commit-msg
-
- Feb 07, 2022
-
-
Antoine R. Dumont authored
Related to T3916
-
- Jan 21, 2022
-
-
vlorentz authored
-
- Jan 11, 2022
-
-
vlorentz authored
A future release of swh-model will change its constructor's signature (replace 'offset' and 'negative_utc_offset' with 'offset_bytes).
- Dec 21, 2021
-
-
Vincent Sellier authored
Related to T3815
-
- Dec 16, 2021
-
-
Antoine R. Dumont authored
This also: - drops spurious copyright headers to those files if present. - drops a no longer relevant type ignore - add another type ignore (to fix later) Related to T3812
-
- Jun 09, 2021
-
-
Antoine R. Dumont authored
-
- May 28, 2021
-
-
Antoine R. Dumont authored
This fixes error instances [1] [1] https://sentry.softwareheritage.org/share/issue/a5869f724a244fdab2e1cb0303a12fc1/
-
Antoine R. Dumont authored
This fixes occurrences of [1] [1] https://sentry.softwareheritage.org/share/issue/4b92519efe2a4ddd9455f0d600a614b9/
-
- May 06, 2021
-
-
vlorentz authored
It renamed db_name to dbname, which is a breaking change.
-