Needed by the backfiller (because it is needed to replay)

It has been observed calling that method can lead to cassandra
query timeout when a snapshot contains a large amount of branches.

The performance bottleneck seems to be the use of a custom cassandra
aggregate function computing the number of branches per target type.

Removing the use of that function and performing the aggregation
client side significantly improves query performance and makes
the timeout issue goes away.

Fixes #4719.

The assertion was failing when executing the code so use a different
one to keep mypy happy.

Also add test to ensure the diagram generation code is covered.

This should speed up a bit tests execution on Jenkins.

Use pytest option --dist=loadfile to group tests by their containing file
in order to avoid spawning too many cassandra or postgres instances.

Recent changes in swh-storage added the PartitionsManagementInterface
but this interface is only implemented on real storage backends, not
proxies.

Any client code overriding the swh_storage_backend_config to return a
storage pipeline (loaders for instance) ends up with error otherwise.

Through a `table_option` dict in the config, mapping table names to what to add to the `WITH` clause of the `CREATE TABLE` clause.

This abstracts this interface from the PostgreSQL-specific implementation,
allowing the object_references table to be sharded in Cassandra.

The storage returned by swh_storage() must be either the same as swh_storage_backend(), or
be a chain of proxies to it. This allows tests to act on the backend using methods
not available on proxies or the RPC

Check these 2 backends can be initialized with `swh db` commands using
both the backend syntax (<package>:<cls>) and the full module name (eg.
storage.proxies.masking).

This requires swh.core 3.6.1 which includes a fix for these.

instead of specific masking_db and blocking_db.
Keep support for these but mark them as deprecated.

Not all codepaths are tested with 3.9, but at least `__init__.py` should work.

This tool can then be used by the sync script filling this db from the
content of the displayname management app in the web app.

Fix broken tests due to the introduction of the ModelObjectType enum.

Closes #4712

For consistency with other *_get methods in this interface, set
return type to list instead of iterable, plus it enables to use
the __getitem__ method in client code without mypy errors.

Latest tenacity release adds some internal changes that broke the
mocking of sleep calls in tests.

Fix it by directly mocking time.sleep (was not working previously).

It is fairly easy to overlook calling `storage.flush()` after adding
some objects just before the end of a process. In a direct
configuration, everything will be fine but if a buffering proxy is
configured, this means that the objects still in the buffers are going
to be lost. Let’s add a warning if this is the case.

When deleting a snapshot, deletion removed the corresponding entries in
the `snapshot_branch` table. These entries are shared between
snapshots (e.g. all snapshots with a `HEAD` pointing to `refs/heads/main`
share the same line in this table), and we don't keep a reverse index to
efficiently remove orphan lines, so we just skip the removal altogether.

Add a test to ensure that the corruption doesn't happen anymore.

These changes are needed to allow `swh alter remove` to remove
RawExtrinsicMetadata and ExtID objects from the PostgreSQL and Cassandra
implementation of the storage.

As RawExtrinsicMetadata objects can be addressed using an
ExtendendSWHID, they can now be removed using `object_delete()`.

For ExtID, a new method `extid_delete_for_target()` has been added to
`ObjectDeletionInterface`. It will remove any ExtID object targeting one
of the object in the a list of SWHID given as an argument.

Address swh-alter!11

The cursor provided by the `@db_transaction` decorator was not
being passed properly.

To be able to use the `swh db` command line, the configurations need to
have the following shape:

<toplevel key>:
  cls: postgresql
  db: <postgresql DSN>

The usage of `blocking_db` and `masking_db` instead of plain `db`, and
lack of overridability in the swh db utilities, made schema migrations
more annoying than necessary.

The old configuration key still works but raises a Deprecation warning.

The configuration of the proxies themselves is unchanged, as having a
differently-named key for the proxy-specific database makes sense to
avoid confusion.

The prefix matching semantics rely on URL patterns not ending with a /.
URLs ending in / will only be used for exact matches. Warn the operator
of this situation.