This also simplifies the db collaborator code reusing core.db functions to check the
code version and the actual db version matches.

Related to T4305

instead of swh-core's postgresql_fact one, since we actually do not
use its custom features any more in swh-storage.

It will be used i swh-storage to fetch a complete directory object,
ie. with the raw_manifest and all branches.

That hook can be frustrating as it can discard a long commit message
if it finds a typo in it so better removing it.

This will make the sentry reports more usable. If the exception changes
across calls, the earlier exceptions are still logged and available as
breadcrumbs.

When using an inner join for the single origin value, instead of a
subquery, the query fails to use the (origin, visit) indexes and falls
back to fetching all the visits (or all the statuses) for the origin and
sorting them.

This breaks down for origins with a lot of visits, such as the ones that
are being used for end to end monitoring.

Using a subselect to generate a single origin id value ensures that the
queries can use the proper indexes.

Align RPC endpoint path with method name.

Ensure to return an empty list of results when fetched visits list is
empty, for instance when the provided page_token is greater or equal
to latest origin visit.

Related to T4090

Related to T3922

black is considered stable since release 22.1.0 and the version
we are currently using is quite outdated and not compatible with
click 8.1.0, so it is time to bump it to its latest stable release.

Please note that E501 pycodestyle warning related to line length
is replaced by B950 one from flake8-bugbear as recommended by black.
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Related to T3922

It enables to retrieve in an efficient and paginated way the list
of visits and all their statuses for a given origin.

Previously, it was required to call origin_visit_status_get on each
visit of the origin to get such list.

Related to T4090

pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for
pytest >= 7 so we can now drop the pytest pinning.

Adding a `query_options` member to postgresql's Storage, in conjunction
with swh.core >= 2.5, allows to set/overwrite SQL client options from
the storage configuration file.

Default values are set, as they used to be, from the decorator
arguments. But in addition to this, one can overload these value at run
time from the storage configuration file. For example:

.. code-block:: yaml

   storage:
     cls: postgresql
     db: testdb
     objstorage:
       cls: memory
     query_options:
       directory_ls:
         statement_timeout: 180000

will provide a Storage instance for which the timeout value for the
`directory_ls` endpoint is 3mn (instead of the default 20s).

The logic for testing the tenacious storage proxy by interspersing bad
objects with good ones triggers when the object list is larger than 3
items. However, the allowed error rate of 1 failure for a window of 3
objects would only work for lists larger than six objects (putting at
least 2 good objects between each bad object).

swh.model 6 made the directory, revision and release test object lists 3
object long, triggering the buggy code.

Reducing the window size to 2 objects makes the test logic works for
lists of 3 or more objects.

swh-model 6.0.0 adds proper support for them.

+ fix issue found by mypy

committer=None happens on some malformed commits generated by old dgit
version; and it is possible for author=None to happen for the same reason.

For now, this is not supported by swh-model, so tests temporarily
disable attrs checks that swh-model relies on.

Due to test modules being copied in subdirectories of the
build directory by setuptools, it makes pytest fail by raising
ImportPathMismatchError exceptions when invoked from root
directory of the module.

So ignore the build folder to discover tests.

This was not covered by tests so far, because
swh.model.tests.swh_model_data.TEST_OBJECTS did not contain
any object with a raw_manifest.

But it will in swh-model > 5.0.0

Without the type annotation, mypy errors with 'Cannot call function of unknown type'
when called from a type-checked function.

This allows both the postgresql and cassandra backends to make efficient
queries by using an index (resp. clustering key) instead of scanning
all visits of the given origin then sorting by date.

This does not affect the results for the last majority of cases,
as ids are always in increasing chronological, unless an origin was
re-loaded from an old archive.

postgresql's query planner does not understand the origin is unique, so it performs
a partial index scan on origin_visit_pkey, which is inefficient on origins with
many visits.

This commit itself is not enough to make it use the proper index,
but provides this necessary change that will be used by a future commit.

According to Sentry, in the last 30 days:

* directory_entry_get_by_path: 958 events, https://sentry.softwareheritage.org/share/issue/c4c2124953a145b2bd325f6f6b7df5a6/
* revision_get: 841 events, https://sentry.softwareheritage.org/share/issue/55fbe01c6f4d4c9bbf684c7608a62ad9/
* release_get: 14 events, https://sentry.softwareheritage.org/share/issue/37c53354541b4c4eaa1faf4e20a68418/
* origin_visit_find_by_date: 114 events, https://sentry.softwareheritage.org/share/issue/a674c12049a941968a717661a0226559/
* origin_get: 79 events, https://sentry.softwareheritage.org/share/issue/bf21d6bc7b24442eb18643d80d936d27/ ; 67 events, https://sentry.softwareheritage.org/share/issue/010a4b1e085a4e2089ba4897c6de6038/

it's not used by swh.storage.

Motivation: replaces code duplication in the backends with a single one,
to be consistent with the objstorage (which has many more backends)

This also fixes the issue of metrics from 'extid_add' to be missing
when using the postgresql storage.

- Add expected entry points for swh.core 2 db handling new features:

  - add a ``swh.storge.get_datastore()`` function
  - add ``swh.storage.postgreql.storage.Storage.get_current_version()`` method
  - move sql migration scripts in ``swh/storage/sql/upgrades``
  - modify sql initialization scripts to match swh.core 2 (remove
    dbversion management code).

- Update tests to use the new template-based database handling; this
  should have only minimal impact on test execution performances.

To install the new hook:

  $ pre-commit install -t commit-msg

I somehow forgot to stage this change in the previous commit.

This is needed by the vault.

Extend the APIs for Revisions and Releases to honor the field by default,
unless the new `ignore_displayname` argument is set.