- Oct 18, 2021
-
-
Jenkins for Software Heritage authored
-
Jenkins for Software Heritage authored
Update to upstream version '0.18.2' with Debian dir 064dff3f140dad4fe33f0893b48748460076b59a
-
Antoine R. Dumont authored
This actually fixes the debian build failure. Related to T3666
- Oct 15, 2021
-
-
Jenkins for Software Heritage authored
Update to upstream version '0.18.1' with Debian dir 33b5f1325f7a752172f38a90c9be255f3bac2402
-
Antoine R. Dumont authored
This scenario happens with the loader oneshot for example. This loader deals with more than 1 type of origins to ingest in the same queue. So the computation of that function returned negative value [1]. Which is ultimately not possible to execute in sql [1]. This commits fixes that behavior. This also explicits that the function must return positive values in its docstring. [1] ``` ... psycopg2.errors.InvalidRowCountInLimitClause: LIMIT must not be negative ```
- Sep 02, 2021
-
-
Jenkins for Software Heritage authored
-
Jenkins for Software Heritage authored
Update to upstream version '0.18.0' with Debian dir e98b6b91a4c915d0ad9f6ee1286273ed99ee3b5b
-
Antoine R. Dumont authored
- Aug 27, 2021
-
-
Antoine R. Dumont authored
In the non optimal case, we may want to trigger specific case (not-yet enabled origins, origin from specific lister...). Related to T3350
-
- Aug 26, 2021
-
-
Nicolas Dandrimont authored
For origins that have never been visited, and for which we don't have a queue position yet, we want to visit them in the order they've been added.
-
Nicolas Dandrimont authored
The subcommand bypasses the legacy task-based mechanism to directly send new origin visits to celery
-
Nicolas Dandrimont authored
Running common operations on all git origins is pretty intense. Using table sampling gives us the opportunity to at least schedule some jobs in (decently small) time.
-
Antoine R. Dumont authored
-
Jenkins for Software Heritage authored
-
Jenkins for Software Heritage authored
Update to upstream version '0.17.1' with Debian dir e98b6f8fc3c8547ef7148fd0b9915f432584ab81
-
Antoine R. Dumont authored
Queue positions are date and the current next_position_offset used to compute the new queue position was not bounded. This has the side-effect of making overflow error. This commit adapts the journal client computations to limit such next_position_offset to 10. This value was chosen because above that exponent the dates overflow (and we are way in the future already). Related to T3502
-
- Aug 18, 2021
-
-
vlorentz authored
We changed the task name/interface a while ago
-
- Aug 06, 2021
-
-
Jenkins for Software Heritage authored
-
Jenkins for Software Heritage authored
Update to upstream version '0.17.0' with Debian dir c55959218dbe52af5f47cb3541419dfb69d77945
- Aug 03, 2021
-
-
Antoine R. Dumont authored
This disable origins for either failed or not found attempts 3 times in a row. It's not definitive though as it's the lister's responsibility to activate back origins if they get listed again. Related to T2345
-
This maintains the number of successive visits resulting in the same status. This will help implementing disabling of too many successive failed or not_found visits for a given origin. Related to T2345
-
- Jul 30, 2021
-
-
Antoine R. Dumont authored
Related to T2345
-
Antoine R. Dumont authored
This is no longer required as it's called once. Related to T2345
-
- Jul 23, 2021
-
-
Nicolas Dandrimont authored
After using this schema for a while, all queries can be implemented in terms of these two timestamps, instead of the four original last_eventful, last_uneventful, last_failed and last_notfound timestamps. This ends up simplifying the logic within the journal client, as well as that of the grab_next_visits query builder. To make this change work, we also stop considering out of order messages altogether in journal_client. This welcome simplification is an accuracy tradeoff that is explained in the updated documentation of the journal client: .. [1] Ignoring out of order messages makes the initialization of the origin_visit_status table (from a full journal) less deterministic: only the `last_visit`, `last_visit_state` and `last_successful` fields are guaranteed to be exact, the `next_position_offset` field is a best effort estimate (which should converge once the client has run for a while on in-order messages).
-
Antoine R. Dumont authored
Related to D5917
-
Antoine R. Dumont authored
This simplifies and unifies properly the utility test function to compare visit stats.
-