Skip to content

backfill: only flush the journal writer on every batch

This module's use of write_addition predated the introduction of reliable writing in swh.journal; Since this introduction, the backfiller has been flushing the kafka writer after writing each single object, leading to a 3x measured slowdown on backfilling contents.

Test Plan

toxed and tested in production on getty


Migrated from D4471 (view on Phabricator)

Merge request reports