Skip to content
Snippets Groups Projects
Forked from Platform / Development / swh-provenance
Source project has a limited visibility.
  • David Douard's avatar
    e66a2bf9
    Add a backfiller cli command · e66a2bf9
    David Douard authored and Nicolas Dandrimont's avatar Nicolas Dandrimont committed
    This command allowd to backfill a kafka journal from an existing
    Postgresql provenance storage.
    
    The command will run a given number of workers in parallel. The state of
    the backfilling process is saved in a leveldb store, so interrupting and
    restarting a backfilling process is possible, with limitations: it won't
    work properly if the range generation is modified.
    e66a2bf9
    History
    Add a backfiller cli command
    David Douard authored and Nicolas Dandrimont's avatar Nicolas Dandrimont committed
    This command allowd to backfill a kafka journal from an existing
    Postgresql provenance storage.
    
    The command will run a given number of workers in parallel. The state of
    the backfilling process is saved in a leveldb store, so interrupting and
    restarting a backfilling process is possible, with limitations: it won't
    work properly if the range generation is modified.