Add an objstorage checker to detect missing and corrupted contents

Add class ObjectStorageChecker to detect missing and corrupted contents in an object storage.

It iterates on content objects referenced in a storage instance, check they are available in a given object storage instance then retrieve their bytes from it in order to recompute checksums and detect corruptions.

Some refactorings have also been done in a first commit to extract checkers common code and features in abstract base classes.

Depends on swh-objstorage!177 (merged).

Related to #4694 (closed).

changed the description

Jenkins job DSCRUB/gitlab-builds #163 failed .
See Console Output and Coverage Report for more details.

added 1 commit

• 900fa193 - Add an objstorage checker to detect missing and corrupted contents

Compare with previous version

^ Add utility function to create a Datastore from an object storage configuration

Jenkins job DSCRUB/gitlab-builds #164 failed .
See Console Output and Coverage Report for more details.

lgtm

approved this merge request

added 2 commits

• 4e834396 - journal_checker: Remove use of deprecated stop_on_eof client parameter
• 651f647a - Add an objstorage checker to detect missing and corrupted contents

Compare with previous version

resolved all threads

Jenkins job DSCRUB/gitlab-builds #165 succeeded .
See Console Output and Coverage Report for more details.

approved this merge request

merged