- Aug 20, 2020
-
-
Antoine Lambert authored
They are not of interest anymore as swh-web is the only client and now directly uses functions defined in swh.storage.algos.diff.
-
vlorentz authored
They got copied to test_postgresql.py instead of moved.
-
Antoine Lambert authored
Also move the round_to_milliseconds function to the storage.utils module.
-
- Aug 19, 2020
-
-
vlorentz authored
-
vlorentz authored
Fixes a regression in 038a219f, as it made the converter expect a list. When adding this column, we made it default to null instead of defaulting to an empty array, so existing records were initialized will null. This commit migrates these nulls to empty arrays, then adds a constraint to enforce it in the future.
-
vlorentz authored
-
vlorentz authored
Fixes a regression in 038a219f, as it made converters expect a boolean. We stopped writing this kind of nulls since we started using model objects for insertions, but didn't migrate existing data. This commit migrates these nulls to false, then adds a constraint to enforce it in the future.
-
vlorentz authored
-
vlorentz authored
This field wasn't backfilled; and it wasn't caught by tests, because revisions in swh.journal.tests.journal_data.TEST_OBJECTS were had an empty extra_header field. Starting with swh-journal v0.4.3, one of the revisions in TEST_OBJECTS has a non-empty extra_header field, so it will become a test failure without this commit
-
- Aug 17, 2020
-
-
vlorentz authored
As a temporary workaround to remain compatible with existing backfiller converters, I made them convert back to dict before they are converted again to model objects. This commit removes this workaround by making converters return model objects.
-
vlorentz authored
This is a change internal to the pg storage, that will be needed to make revision_get, revision_log, and release_get return model objects.
-
vlorentz authored
-
- Aug 14, 2020
-
-
vlorentz authored
in the two methods which need to switch between statements. The method name building was done to shoehorn this statement switching into the existing @_prepared_select_statement. This introduces @_prepared_select_statements (plural), which does this switching properly, using a dictionary.
-
vlorentz authored
It's not optional in the storage interface, so a None value can't be passed.
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
It's not used anywhere.
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
-
vlorentz authored
in_memory: Remove InMemoryStorage.skipped_content_* and implement InMemoryCqlRunner.skipped_content_*
-
vlorentz authored
-
vlorentz authored
For now this has no effect. However, in the near future, the CassandraStorage will be in charge of some object types, so we need to merge objects found in the CassandraStorage and those found directly in the InMemoryStorage.
-
vlorentz authored
For now it's only used for object counters; but future commits will progressively move the in-memory's storage features to it.
-
vlorentz authored
This has no effect for now, other than deduplicating a method and causing a name clash.
-
vlorentz authored
It will be used to implement the in-memory storage as a backend for the cassandra storage.
-
vlorentz authored
-
vlorentz authored
They will be used by the in-mem implementation of CqlRunner.
-
vlorentz authored
cassandra: Make origin_visit_get_latest filter using any status of a visit, instead of just the last. This fixes a mismatch in behavior with the pg and the in-mem storages
-
vlorentz authored
-
vlorentz authored
-
- Aug 11, 2020
-
-
vlorentz authored
'SELECT *' should be avoided in prepared statements, see https://docs.datastax.com/en/developer/java-driver/3.0/manual/statements/prepared/#avoid-preparing-select-queries
-
vlorentz authored
cassandra: add a TABLE class attribute to row classes, and use it to deduplicate prepared statements logic. It will also be used in a future commit to generate 'select' prepared statements.
-
vlorentz authored
All methods of CqlRunner were decorated, which prevented mypy from doing anything useful. As I finally found a way to type the decorator (using mypy_extensions.NamedArg), I can finally make mypy aware of the methods' types. This commit (as well as all three of the last commits) also fixes issues found by mypy thanks to this.
-
vlorentz authored
-