- Jan 06, 2023
-
-
Kartik Ohri authored
-
Kartik Ohri authored
Summary: Add type annotations to files in swh/indexer/metadata_dictionary/ (T2259) This patch adds type annotations to metadata mappings so that the mypy type checker can detect bugs in code. While working on the path, I also encountered these issues: 1) TypedDict does not support using variables in defining or assigning keys. At a lot of places, the code uses SCHEMA_URI to construct the keys. mypy raises errors at these places. There is a tradeoff between replacing SCHEMA_URI with its literal value everywhere and using type: ignore hints. 2) mypy raises error on detecting for implicit None returns, so I had to add return None at many places. 3) At a few places, @property s' are accessed using `cls`. mypy rightfully raises an error, so I had to add ignore to silence it for now. It appears Python 3.9+ supports class level properties so the ignores could be replaced by the proper handling when the minimum python version is raised. Closes T2259. Reviewers: vlorentz
-
- Feb 24, 2022
-
-
David Douard authored
- add the `get_datastore` function in `swh.indexer` - add the `get_current_version` method in `IndexerBackend`, - remove dbversion management from sql init script - update tests accordingly
-
vlorentz authored
-
- Feb 10, 2022
-
-
Antoine Lambert authored
To install the new hook: $ pre-commit install -t commit-msg
-
- Feb 07, 2022
-
-
Antoine R. Dumont authored
Related to T3916
-
- Jan 21, 2022
-
-
vlorentz authored
-
- Jan 11, 2022
-
-
vlorentz authored
A future release of swh-model will change its constructor's signature (replace 'offset' and 'negative_utc_offset' with 'offset_bytes).
- Dec 21, 2021
-
-
Vincent Sellier authored
Related to T3815
-
- Dec 16, 2021
-
-
Antoine R. Dumont authored
This also: - drops spurious copyright headers to those files if present. - drops a no longer relevant type ignore - add another type ignore (to fix later) Related to T3812
-
- Jun 09, 2021
-
-
Antoine R. Dumont authored
-
- May 28, 2021
-
-
Antoine R. Dumont authored
This fixes error instances [1] [1] https://sentry.softwareheritage.org/share/issue/a5869f724a244fdab2e1cb0303a12fc1/
-
Antoine R. Dumont authored
This fixes occurrences of [1] [1] https://sentry.softwareheritage.org/share/issue/4b92519efe2a4ddd9455f0d600a614b9/
-
- May 06, 2021
-
-
vlorentz authored
It renamed db_name to dbname, which is a breaking change.
-
- Apr 26, 2021
-
-
Antoine Lambert authored
Enable to check package documentation can be built without producing sphinx warnings. The sphinx environment is designed to be used in continuous integration in order to prevent breaking documentation build when committing changes. The sphinx-dev environment is designed to be used inside a full swh development environment. Related to T3258
-
- Apr 02, 2021
-
-
Kumar Shivendu authored
The implementation is based on output from cff-converter python library. Deviation has been made for license and organisation.legalName(replaced with organisation.name)
-
- Mar 19, 2021
-
-
Kumar Shivendu authored
codemeta dir was updated last time in Jan 2019. In this commit crosswalk.csv, CITATION and codemeta.jsonld have been updated.
-
- Mar 06, 2021
-
-
Kumar Shivendu authored
-
- Mar 01, 2021
-
-
Kumar Shivendu authored
Add testcase to detect CODEMETA.json
-
- Feb 01, 2021
-
-
Antoine R. Dumont authored
-
vlorentz authored
self._index_contents was called multiple times in a loop with the same arguments, except for the set of hashes to exclude. It means that, if there were N pages of hashes to exclude, each content was indexed N times; and the N-1 first iterations didn't even exclude all the hashes they had to exclude.
-
- Jan 04, 2021
-
-
David Douard authored
-
- Dec 04, 2020
-
-
Antoine R. Dumont authored
The reason for this is to avoid surprises like the indexer journal client stuck in limbo for a while. Related to T2821 Related to T2814
-
- Dec 02, 2020
-
-
Antoine R. Dumont authored
Related to D4638
-
Antoine R. Dumont authored
This detected some paper cuts within cli tests for example. The main goal is to decrease friction when actually deploying indexer related services (backend, indexers, ...). The pg backends tests should still be reasonably fast as it's using the swh.core.db.pytest_plugin (which truncate tables in between tests). Related to T2821
-
- Nov 27, 2020
-
-
Antoine R. Dumont authored
According to the value_sanitizer docstring, this takes 2 parameters, first is the object type, the second is the actual dict value to sanitize. As a somewhat default identity function, this discards the object type and returns directly the dict value unchanged. [1] https://forge.softwareheritage.org/source/swh-journal/browse/master/swh/journal/writer/kafka.py$97-100
-
vlorentz authored
This always happens when writing to Kafka, as the Kafka writer sets it to None at the same it it injects the 'tool' data. This was not caught by tests because they use the in-mem writer; which did not call unique_key() at all in swh-journal<=v0.5.1 (but future versions will).
-
Antoine R. Dumont authored
This fixes the indexer debian package build.
-
Antoine R. Dumont authored
It's simplifying reading and more consistent with other similar tests
-
- Nov 26, 2020
-
-
Antoine R. Dumont authored
-
Antoine R. Dumont authored
... instead of OriginVisit. OriginVisit model object no longer hold status information so the current filtering happening on the journal client side could not work. Related to T2814 Related to P882
-
Vincent Sellier authored
The minimum configuration is provided either by the --config-file or the --broker parameters Related to T2814
-
- Nov 16, 2020
-
-
Nicolas Dandrimont authored
swh.storage and swh.objstorage, as well as swh.indexer itself, have deprecated using an explicit `args` in their factories for a while; we can drop them now.
-
Nicolas Dandrimont authored
vcversioner was already removed months ago.
-
- Nov 10, 2020
-
-
vlorentz authored
-
- Nov 05, 2020
-
-
vlorentz authored
postgresql kindly returns the results in the order the test expected... most of the time.