i.e. make it aware of modename given as '<packahge>:<cls>', as they are
now stored in the dbmodule table.

Fall back if not entry point is registered for the package.

and use it from populate_database_from_packae() so it's fixed when
running commands like `swh db init` and `swh dn updrade`.

It is IMHO easier to read.

Allow db commands (create, init-admin, init, version, upgrade) get the
db connection string from anywhere in the config file.
For example, it can be used like:

$ cat conf.yml
storage:
  cls: pipeline
  steps:
    - cls: masking
      masking_db: postgresql:///?service=swh-masking-proxy
    - cls: buffer
    - cls: postgresql
      db: postgresql://user:passwd@pghost:5433/swh-storage
      objstorage:
        cls: memory

$ swh db -C conf.yml init storage:steps:0:masking_db

This de facto deprecates the usage of the --module-config-key option.

Also add a -a/--all option to these commands allowing to execute the given
command to all matching configuration entries in the config file; this
should help deployment in testing environments like docker.

For example:

  $ swh db version -a storage

  module: storage.proxies.masking
  current code version: 194
  version: 194

  module: storage
  flavor: default
  current code version: 193
  version: 193

This change also make the module name stored in the dbmodule table an
fully qualified module instead of shortened version, i.e. it will now
store 'swh.storage.postgresql.storage' instead of just 'storage',
so there is less "implicit logic" (like removing "swh.", etc.)
when filling/using this piece of information.

We add a bw compat support in get_sql_for_package to ease the transition
until the dbmodule tables are updated. This bw compat logic will
look for the 'sql/' directory not only in the package directly for the
module, but also in parent directories. This allows to keep the code
compatible with current version of swh.storage (in which the sql/
directory and the actual backend submodule do not match).

Doing this, we also stop defining a hardwritten dbname ('softwareheritage-dev')
as default value for the '--dbname' option in 'swh db create' command.

Warning: for the 'create' and 'init-admin' db commands, looking for the
connection string in the config file is mainly useful for test environments
in which the user used to access the backend database have superuser
permissions on the database. In production-like environments, you most
certainly will prefer using an explicit --dbname argument with a
connection with enough permission to handle the job.

Show the full traceback of the exception that occurred in the cli_runner
execution, if any.

It better captures the semantic of the option and makes the '--all'
option free for a common usage to handle the 'operate on all db backends
in the configuration file' semantic (i.e. replace '--initialize-all' and
'--upgrade-all' options) in the future.

importlib.metadata.entry_points only supports the selection behavior
in python 3.10+.

When running command line tools like `swh alter` or `swh scanner`, there
is rarely a Sentry DSN set. As errors are most often monitored from the
terminal in such configuration, let’s not display a warning that is more
noise than signal.

`.called_once_with` doesn't exist, so it was just returning a mock
object which was boolean True; however pytest-mock is now catching
this (correctly) as a programming error, so handle the call args of the
exception capture properly.

YAML mappings are converted in Python as dict. The order
of the keys in dicts returned by `load_from_envvar()` were
not stable from one Python run to the next.

Since Python 3.6, dicts are ordered by default, and `yaml.safe_load()`
will always return keys in the same order. The culprit was
`merge_configs` which created a set (which are unordered) of keys
to perform the merge.

Latest tenacity release adds some internal changes that broke the
mocking of sleep calls in tests.

Fix it by directly mocking time.sleep (was not working previously).

This avoids having to use two separate configuration files for
initializing and then running a storage instance.

This will ease database introspection from a pod toolbox without the need to configure
extra specific postgresql.

Co-authored-by: Nicolas Dandrimont <olasd@softwareheritage.org>

We now have dotted modules whose section in the configuration file no longer match their
name. The `db upgrade` cli had been evolved to deal with it but not the `db init` cli.

This should mitigate errors related to streamed HTTP responses.

Migration to importlib introduced a bug when loading swh cli
entrypoints: the entry point group name must be given with a named
argument.

There are still situations where the swh root namespace is not properly
handled when running tests from pytest in an editable-mode installed
package situation, so make the 2 concerned tests pass in both cases.

Also replace a few `assert data.items() <= {xxx: xxx}.items()' idioms by
easier to read and understand statements.

pkg_resources is now considered as deprecated.

Add same option as in 'swh db init-admin' and 'swh db init' commands.

It is quite useful to quickly test database migration locally.

Make the function returns False when such exception is raised,
for instance when a .swh file exists in home directory.

It fixes pre-commit mypy check in swh-web for such edge cases.

Includes support for converting two-value timeout arguments into tuples,
as generating this argument with yaml will provide a list, which
requests doesn't support.

When providing a None DSN to sentry_sdk.init, sentry client is initialized
and functional but the events are not sent.

This type of configuration is useful when running tests so ensure to always
initialize sentry in swh.core.sentry.init_sentry regardless of the dsn value.

sentry-python 2.0 introduced some breaking API changes so update the sentry
fixtures used in tests to restore previous behavior.

Remove use of --import-mode=importlib pytest option and use
new option consider_namespace_packages to fix tests execution
with latest pytest release.

Related to swh/meta#5075.

The db pytest plugin was deprecated in June 2022 (in commit
4fc5f601) and has no users left.

This is the last blocker to supporting pytest-postgresql v5.

To make it easier to debug code in the Docker stack, allow to set the
general log level through the `SWH_LOG_LEVEL` environment variable.
This can then easily be set in `docker.override.yml` like so:

    swh-lister:
      environment:
        SWH_LOG_LEVEL: "debug"

Click provides the plumbing to do this easily, see:
https://click.palletsprojects.com/en/8.1.x/options/#values-from-environment-variables

Typing stubs for click and flask are now maintained by upstreams
so remove their outdated typing packages.