Skip to content
Snippets Groups Projects
Commit 9cd7414a authored by David Douard's avatar David Douard
Browse files

Refactor the checker stack

A checker configuration must now be created before being
able to start a checker session. This configuration is stored in the
database and consist in a triplet

  (datastore, object_type, nb_partitions)

Once done, any number of checker can be started for this specific
checker configuration; each checher process will check partitions
one by one, using the status stored in the database to get the next
partition number to check on the next iteration.

This allows to dynamically adapt the number of checker processes.

For example, checking the shapshots splitting the hash space in 4096
partitions using 4 parallel workers could be like:

  $ export SWH_CONFIG_FILENAME=config.yml
  $ swh scrubber check init --object-type snapshot --nb-partitions 4096 --name cfg-snp
  Created configuration cfg-snp [3] for checking shapshot in postgresql storage

  $ for i in {1..4}; do (swh scrubber check storage cfg-snp &); done
parent 369341bc
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment