- Aug 09, 2022
-
-
vlorentz authored
On the wire, this is done by making the server return a 503 error instead of 500, which the RPC client generated by swh-core interprets to change the exception class.
- Aug 08, 2022
-
-
vlorentz authored
It needs to be linked from swh/lister/crates/__init__.py
-
- Aug 05, 2022
-
-
vlorentz authored
-
vlorentz authored
The postgresql implementation uses '3 months', which is closer to 13 weeks than to 12 weeks.
-
vlorentz authored
start is increased from 13 to 14, because 13 weeks is 91 days, ie. 30+31+30; so it is sometimes smaller than 3 months. This was only hit rarely because the number of visits was small, so this commit also increases the number of visits to make the test more likely to fail if it should actually fail.
- Aug 04, 2022
-
-
vlorentz authored
They are very noisy, and clients are expected to retry a few times before re-raising the exception on their side.
-
vlorentz authored
This caused the following warning: ``` WARNING cassandra.protocol:libevreactor.py:361 Server warning: `USE <keyspace>` with prepared statements is considered to be an anti-pattern due to ambiguity in non-qualified table names. Please consider removing instances of `Session#setKeyspace(<keyspace>)`, `Session#execute("USE <keyspace>")` and `cluster.newSession(<keyspace>)` from your code, and always use fully qualified table names (e.g. <keyspace>.<table>). ``` This also prepends 'test' to the name of keyspaces used in tests, so they are guaranteed to start with an letter (starting with digits cause syntax errors in most statements).
-
vlorentz authored
-
vlorentz authored
This reproduces what I think is the issue found in https://jenkins.softwareheritage.org/job/debian/job/packages/job/DSTO/job/gbp-buildpackage/423/consoleFull This does not fix the issue as it is a consequence of the design, but documents this problematic behavior.
-
- Jul 13, 2022
-
-
Antoine Lambert authored
Even if missing index to speedup origin visit queries has been added to replica database, the configured timeouts for origin_visit_get_with_statuses and origin_visit_find_by_date were still too low to avoid query timeouts in production. After performing some tests locally, bumping them to 2000ms makes the timeouts go away. Related to T4386
-
- Jul 12, 2022
-
-
vlorentz authored
This uses Directory.from_possibly_duplicated_entries() to mangle entry names instead of crashing.
-
- Jul 08, 2022
-
-
David Douard authored
-
- Jul 06, 2022
-
-
David Douard authored
when the OriginVisit object given as argument to be inserted already have its visit id set (which is usually the case in a replayer-like session), it makes no sense to auto-add the first OriginVisitStatus objects related to this visit; this behavior is expected only when the origin_visit_add() is called from a loading session. Adapt tests accordingly -- several tests did depend on the auto-add behavior of the origin_visit_add method for OriginVisit objects which visit_id is given in the test dataset.
-
- Jul 01, 2022
-
-
David Douard authored
and add tests for 'mirror' and 'read_replica' flavors.
-
David Douard authored
-
- Jun 03, 2022
-
-
Antoine R. Dumont authored
This also simplifies the db collaborator code reusing core.db functions to check the code version and the actual db version matches. Related to T4305
-
- May 31, 2022
-
-
David Douard authored
instead of swh-core's postgresql_fact one, since we actually do not use its custom features any more in swh-storage.
-
David Douard authored
-
- May 10, 2022
-
-
vlorentz authored
-
- May 09, 2022
-
-
Pratyush authored
-
- May 02, 2022
-
- Apr 28, 2022
-
-
Antoine R. Dumont authored
-
- Apr 26, 2022
-
-
vlorentz authored
-
- Apr 21, 2022
-
-
Antoine Lambert authored
That hook can be frustrating as it can discard a long commit message if it finds a typo in it so better removing it.
-
- Apr 14, 2022
-
-
David Douard authored
-
- Apr 13, 2022
-
-
Nicolas Dandrimont authored
This will make the sentry reports more usable. If the exception changes across calls, the earlier exceptions are still logged and available as breadcrumbs.
-
- Apr 12, 2022
-
-
Nicolas Dandrimont authored
When using an inner join for the single origin value, instead of a subquery, the query fails to use the (origin, visit) indexes and falls back to fetching all the visits (or all the statuses) for the origin and sorting them. This breaks down for origins with a lot of visits, such as the ones that are being used for end to end monitoring. Using a subselect to generate a single origin id value ensures that the queries can use the proper indexes.
-
Antoine Lambert authored
Align RPC endpoint path with method name.
-
Antoine Lambert authored
Ensure to return an empty list of results when fetched visits list is empty, for instance when the provided page_token is greater or equal to latest origin visit. Related to T4090
-
- Apr 08, 2022
-
-
Antoine Lambert authored
-
Antoine Lambert authored
Related to T3922
-
Antoine Lambert authored
black is considered stable since release 22.1.0 and the version we are currently using is quite outdated and not compatible with click 8.1.0, so it is time to bump it to its latest stable release. Please note that E501 pycodestyle warning related to line length is replaced by B950 one from flake8-bugbear as recommended by black. https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length Related to T3922
-
- Apr 07, 2022
-
-
Antoine Lambert authored
It enables to retrieve in an efficient and paginated way the list of visits and all their statuses for a given origin. Previously, it was required to call origin_visit_status_get on each visit of the origin to get such list. Related to T4090
-
- Apr 06, 2022
-
-
Antoine Lambert authored
pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for pytest >= 7 so we can now drop the pytest pinning.
-
- Apr 04, 2022
-
-
David Douard authored
Adding a `query_options` member to postgresql's Storage, in conjunction with swh.core >= 2.5, allows to set/overwrite SQL client options from the storage configuration file. Default values are set, as they used to be, from the decorator arguments. But in addition to this, one can overload these value at run time from the storage configuration file. For example: .. code-block:: yaml storage: cls: postgresql db: testdb objstorage: cls: memory query_options: directory_ls: statement_timeout: 180000 will provide a Storage instance for which the timeout value for the `directory_ls` endpoint is 3mn (instead of the default 20s).
-
- Mar 23, 2022
-
-
Nicolas Dandrimont authored
The logic for testing the tenacious storage proxy by interspersing bad objects with good ones triggers when the object list is larger than 3 items. However, the allowed error rate of 1 failure for a window of 3 objects would only work for lists larger than six objects (putting at least 2 good objects between each bad object). swh.model 6 made the directory, revision and release test object lists 3 object long, triggering the buggy code. Reducing the window size to 2 objects makes the test logic works for lists of 3 or more objects.
-
vlorentz authored
swh-model 6.0.0 adds proper support for them. + fix issue found by mypy
-