Parallel loaders performances
In order to simulate the cluster behavior with an usual swh load, a test should be prepare to have a ingestion infrastructure with a lot of loader running in parallel
Some adaptions need to be done on the bench lab to have a central swh-storage responding to the workers. A fake objstorage needs to be added to handle the worker adds and of course all the ingestion services, a lister / scheduler / rabbitmq ...
The docker infrastucture will probably be used (or perhaps kubernetes as grid5000 provides a k8s deployment environment)
Migrated from T3577 (view on Phabricator)
Edited by Vincent Sellier