Skip to content

Rework the retry and reporting system in replay.py

David Douard requested to merge generated-differential-D6692-source into generated-differential-D6692-target

split the 'copy_object()' in 2 parts, 'get_object()' and 'put_object()' and make each of these decorated by '@content_replay_retry' (instead of the whole 'copy_object()'), so that a failing 'put_object' does not trigger getting the object from the src objstorage again.

Also, only log in statsd the end result (error or success) instead of logging each attempt. We don't need these intermediate results in statsd, and it makes them much harder to use in a dashboard.

Last, for the sake of implementation ease, use function name as "operation" tag in stastds reports (so "get_object", "put_object" and "obj_in_storage") instead of "copy" and "in_dst".

Doing so, the 'operation' argument of the ReplayError exception has been dropped.

Add doctrings and comments in test_cli.py

Small code refactoring in test_cli: make '_fill_objstorage_and_kafka()' take the actual objstorage as argument, instead of the dict of objstorages ("src", "dst"). The "dst" objstorage is not used in the function, and it make the intent of this later clearer.

Migrated from D6692 (view on Phabricator)

Merge request reports