To run these tests on the ceph backend, we actually need to release
shards so they can be unmapped (the file-based backend can just do
whatever to a file that is being held open, but a block device needs
more care). The newly run test also unearthed a small file descriptor
leak in the swh.perfecthash object deletion paths.

The fixture is used for logging but wasn't being injected.

This avoids a bunch of race conditions when running the normal
objstorage test suite on top of a ceph pool.

This should make it easier to substitute the implementation

Extend tests to support instantiating winery with or without a
throttler.

Move throttler database connection setting to the throttler dict, so
that it can be split from the main database connection (e.g. on
read-only workers)

This reduces contention across parallel processes (if they act on
overlapping shards): a crash can happen if two rbd processes call `rbd
map` on the same shard at the same time.

This will give us the option to map shards in parallel

This allows following the (slow) mapping process more closely

There's no point having multiple throttler instances for the separate
ROShards, it's even counterproductive as it generates a connection pool
for each instance, which in turn generates many threads.

Ref. swh/infra/sysadm-environment#5591

This should help limit the blast radius of future shards pool management
api changes.

This should allow refactorings to pass components only the settings that
they actually need, rather than ignoring all kwargs. It should also help
make the transition towards a configurable shards pool backend more tractable.

We don't expect to be using Grid'5000 again so this code has not been
maintained with expectations that it would work again. This allows us to
streamline future refactorings as well.

A future objstorage benchmark tool could take inspiration from a lot of
this code, but be agnostic to the backend used.

The stats are only used in the benchmark process and will not be
exploited again.

When both hosts try to record the shard as mapped read-only at the same
time, one of them can fail. This situation cannot be recovered from
automatically.

To recover from the situation, first, add a retry to the
record_shard_mapped call, which should make it succeed from within the
main loop; second, when initializing the list of shards, if the shard is
mapped read-only and its state is "PACKED" (which should be pretty
rare), then mark it as mapped.

All our uses of it have been adopted upstream

Rather than repeating a hardcoded list of hashes everywhere.

The call was missing the conversion to sha256 before passing it to the
backend, and delete() must return True if the deletion has happened.

The previous implementation would only raise the exception with a
sha256, which is not what tests expect.

The main goal of this commit is adding support for PRIMARY_HASH==sha256
in the default implementation of list_content. To do so, we need to
introduce PRIMARY_HASH in the ObjStorageInterface, which pulls a bunch
of changes to appease mypy.

Supposedly this might make it easier to handle configurable PRIMARY_HASH?

Some tests rely on this correspondence.