Skip to content

Add some progress bars for long-running operations

Jérémy Bobbio (Lunar) requested to merge lunar/swh-alter:progressbars into main

We introduce a new module swh.alter.progressbar that wraps around what is provided by click.progressbar. While it provides a progressbar() method with a similar interface, it also defines a no_progressbar() method doing no-ops. The latter can be used as a default, while allowing to use the same code when a progressbar is displayed or not.

We might want to revise the values of STORAGE_OBJECT_DELETE_CHUNK_SIZE, RECOVERY_BUNDLE_BACKUP_SWHIDS_CHUNK_SIZE, and RECOVERY_BUNDLE_RESTORE_CHUNK_SIZE depending on the performance hits.

Example output:

Removing origins from search “main”…
  [####################################]  100%
1 origins removed from search “main”.
Removing objects from storage “postgresql”…
431 objects removed from storage “postgresql”.
Removing objects from storage “cassandra”…
431 objects removed from storage “cassandra”.
Removing objects from journal “main”…
Objects removed from journal “main”.
Removing objects from objstorage “extra”…
  [####################################]  100%
204 objects removed from objstorage “extra”.
Removing objects from objstorage “primary”…
  [####################################]  100%
204 objects removed from objstorage “primary”.

Depends !19 (merged)

Edited by Jérémy Bobbio (Lunar)

Merge request reports