idx.storage.JournalWriter: pass value_sanitizer to get_journal_writer.
Closed
requested to merge generated-differential-D4623-source into generated-differential-D4623-target
1 unresolved thread
Merge request reports
Activity
Build is green
Patch application report for D4623 (id=16387)
Could not rebase; Attempt merge onto d87fd531...
Updating d87fd53..73e181b Fast-forward swh/indexer/storage/model.py | 15 +++++++++------ swh/indexer/storage/writer.py | 4 +++- swh/indexer/tests/storage/test_model.py | 26 ++++++++++++++++++++++++++ 3 files changed, 38 insertions(+), 7 deletions(-) create mode 100644 swh/indexer/tests/storage/test_model.py
Changes applied before test
commit 73e181b05510ff2325ab4b64a71d64bd7807e899 Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 27 13:37:14 2020 +0100 idx.storage.JournalWriter: pass value_sanitizer to get_journal_writer. It will become mandatory in a future version. commit 2cca6cbcf0b4fafaefbb2aa1d90b662a3a8a997d Author: Valentin Lorentz <vlorentz@softwareheritage.org> Date: Fri Nov 27 13:34:05 2020 +0100 BaseRow.unique_key: Don't crash when indexer_configuration_id is None. This always happens when writing to Kafka, as the Kafka writer sets it to None at the same it it injects the 'tool' data. This was not caught by tests because they use the in-mem writer; which did not call unique_key() at all in swh-journal<=v0.5.1 (but future versions will).
See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/129/ for more details.
37 37 "You need the swh.journal package to use the " 38 38 "journal_writer feature" 39 39 ) 40 self.journal = get_journal_writer(**journal_writer) 40 self.journal = get_journal_writer( 41 **journal_writer, value_sanitizer=lambda x: x
Please register or sign in to reply