When running command line tools like `swh alter` or `swh scanner`, there
is rarely a Sentry DSN set. As errors are most often monitored from the
terminal in such configuration, let’s not display a warning that is more
noise than signal.

`.called_once_with` doesn't exist, so it was just returning a mock
object which was boolean True; however pytest-mock is now catching
this (correctly) as a programming error, so handle the call args of the
exception capture properly.

YAML mappings are converted in Python as dict. The order
of the keys in dicts returned by `load_from_envvar()` were
not stable from one Python run to the next.

Since Python 3.6, dicts are ordered by default, and `yaml.safe_load()`
will always return keys in the same order. The culprit was
`merge_configs` which created a set (which are unordered) of keys
to perform the merge.

Latest tenacity release adds some internal changes that broke the
mocking of sleep calls in tests.

Fix it by directly mocking time.sleep (was not working previously).

This avoids having to use two separate configuration files for
initializing and then running a storage instance.

This will ease database introspection from a pod toolbox without the need to configure
extra specific postgresql.

Co-authored-by: Nicolas Dandrimont <olasd@softwareheritage.org>

We now have dotted modules whose section in the configuration file no longer match their
name. The `db upgrade` cli had been evolved to deal with it but not the `db init` cli.

This should mitigate errors related to streamed HTTP responses.

Migration to importlib introduced a bug when loading swh cli
entrypoints: the entry point group name must be given with a named
argument.

There are still situations where the swh root namespace is not properly
handled when running tests from pytest in an editable-mode installed
package situation, so make the 2 concerned tests pass in both cases.

Also replace a few `assert data.items() <= {xxx: xxx}.items()' idioms by
easier to read and understand statements.

pkg_resources is now considered as deprecated.

Add same option as in 'swh db init-admin' and 'swh db init' commands.

It is quite useful to quickly test database migration locally.

Make the function returns False when such exception is raised,
for instance when a .swh file exists in home directory.

It fixes pre-commit mypy check in swh-web for such edge cases.

Includes support for converting two-value timeout arguments into tuples,
as generating this argument with yaml will provide a list, which
requests doesn't support.

When providing a None DSN to sentry_sdk.init, sentry client is initialized
and functional but the events are not sent.

This type of configuration is useful when running tests so ensure to always
initialize sentry in swh.core.sentry.init_sentry regardless of the dsn value.

sentry-python 2.0 introduced some breaking API changes so update the sentry
fixtures used in tests to restore previous behavior.

Remove use of --import-mode=importlib pytest option and use
new option consider_namespace_packages to fix tests execution
with latest pytest release.

Related to swh/meta#5075.

The db pytest plugin was deprecated in June 2022 (in commit
4fc5f601) and has no users left.

This is the last blocker to supporting pytest-postgresql v5.

To make it easier to debug code in the Docker stack, allow to set the
general log level through the `SWH_LOG_LEVEL` environment variable.
This can then easily be set in `docker.override.yml` like so:

    swh-lister:
      environment:
        SWH_LOG_LEVEL: "debug"

Click provides the plumbing to do this easily, see:
https://click.palletsprojects.com/en/8.1.x/options/#values-from-environment-variables

Typing stubs for click and flask are now maintained by upstreams
so remove their outdated typing packages.

When building package documentation outside tox by calling make in the
docs directory, the include of Makefile.sphinx inside the docs Makefile
was failing as its relative path was invalid.

So adapt this relative path according if the SWH_PACKAGE_DOC_TOX_BUILD
environment variable is set or not.

Also align black version in tox.ini file.

- remove sphinx-dev tox env,
- use non-editable install of swh-docs to build the sphinx doc in tox,
- remove (now useless) MANIFEST.in.

It's not needed any more.

This replaces the setup.py package file with pyproject.toml (keeping
setuptools and setuptool_scm) and use an implicit namespace declaration
for the swh namespace.

Also note that due to the poor support of PEP 420 in pytest, it needs a
pair of command line arguments to pass all tests (see tox.ini).

Instead of guessing archive format by first checking its extension then
its mime type, prefer to do the opposite as it is more reliable, especially
if an archive file has an incorrect extension.

It's currently happening for some lister, preventing them from finishing their listing.

This adds the missing tests on that part.
This also gives up after 5 attempts.