Skip to content

backfill: only flush the journal writer on every batch

Nicolas Dandrimont requested to merge generated-differential-D4471-source into generated-differential-D4471-target

This module's use of write_addition predated the introduction of reliable writing in swh.journal; Since this introduction, the backfiller has been flushing the kafka writer after writing each single object, leading to a 3x measured slowdown on backfilling contents.

Test Plan

toxed and tested in production on getty

Migrated from D4471 (view on Phabricator)

Merge request reports