This avoids spamming a deprecation error clients can't do anything about.

swh_set_db_module() takes a db_or_conninfo object, and passes it to execute_sqlfiles,
which only expects a conninfo.

This is detected by mypy when using a recent version of types-psycopg2.

This may be needed depending on import order; but it makes mypy complain when
types-psycopg2 is installed.

It will be used by the swh-objstorage client in 'list_content' to efficiently
chunk the output into sha1 digests without relying on HTTP chunking from the server.

This will be used by the objstorage (and possibly the storage as well)
to publish metrics to statsd.

I want to make swh-objstorage inherit from RPCClient, but its get() and get_stream()
methods would class with RPCClient's.

move the import statement in swhmain() body.
Having the swh.core module loaded with the execution of conftest.py can
have weird side effects, especially on pytest's discovery mechanism in
the context of tox, namely make pytest load test files from the source
directory rather than from the installed package in the tox venv,
potentially leading to ImportPathMismatchError.

this is far from completion.

in favor of using (and thus testing) the stock get_sql_for_package()
function (on the mock module returned by mock_import_swhmodule).

This comes with a change in expected test data structure for sql
script; the `tests/data/<mod>` is now expected to store the sql script
files in a dedicated `sql/` sub directory instead of having them
directly in the `<mod>` directory so that the get_sql_for_package() can
be used in tests unmocked.

update the postgresql_fact fixture to be able to use new-ish
pytest_postgresql ability to recreate databases between tests from a
template database.

The ides is to stop using the fragile and confusing "reset" of the
database between tests, which means de facto keep a stateful database
between tests, which make tests result depend on de facto order of
execution of these tests.

For example, test_smoke_test_people_db_up_and_reset cannot be executed
alone, since it depends on the fact test_smoke_test_people_db_up has
been executed before so that the people's db has been reset as a result
of the test setup done by the fixture, which will happen if and only if
the database already exists (thus if another test using this db has been
executed before in this test session).

The old table truncate based behavior has been kept (using a no_db_drop
flag argument) in this diff for bw compatibility reasons, and also because
the new drop-and-create-from-template beahvior remains a bit slower,
which might be a problem for some test cases.

Tests for dependencies (swh-storage, etc.) should still pass with this
refactoring, but will be significantly slower.

A series of revisions for these dependencies is about to be proposed to
adapt them to the new db metadata (version, module, flavor) handling
feature coming with swh.core 2, as well as updating their tests to take
advantage of the template-based db creation in tests, which should bring
back test execution time to almost the same as before this refactoring.

This is needed to be able to implement and test support for the new 'swh
db upgrade' all this is actually about.

this new command allows to handle the database migration from the
current version to the latest version advertized but the backend
datastore.

This feature is only working for "new style" swh.core.db based
datastores.

for this feature to work properly, the backend datastore package must
provide the 'get_datastore' factory function in its root namespace and
the datastore instance must provide a 'get_current_version()' method.

This also means the canonical cli usage for the `swh db init` is now
using the config file (via `--config-file` or `SWH_CONFIG_FILENAME`)
instead of giving only the db cnx string (`--db-name`), so that the
backend datastore class can be instanciated using the `get_datastore()`
factory function.

However a '--initial-version' cli option has been added to 'swh db init'
to make it easier, especially for unit tests.

instead of a custom 'test_db' (which did not truncate dbversion and
origin tables, not sure what the intent was for this).

for now, managing the dbversion table (storing the db schema upgrade
history) is partially the responsibility of the swh modules implementing
a datastore (swh-storage etc.), and the actual swh module was not stored
in the database.

This adds support for both the dbversion and dbmodule tables management
in swh.core.db.

The `swh db init` command now creates both these tables and initialize
their values if possible.
For this, it introduces a new common "API" for swh.core.db based datastores:

- a swh module implementing a datastore is expected to have a
  `get_datastore` function in its top namespace; this function is
  typically the actual `get_storage`, `get_scheduler` etc. functions
- this factory function is expected to use the "postgresql" cls for the
  datastore based on swh.core.db,
- the datastore instance (eg. instance of the swh.storage.postgresql.Storage
  class) is expected to have a `get_current_version()` method (returning
  an int).

This revision also provides a new `swh db version` command displaying
the current (code) version (if available), the db version (possibly the whole
version history) and flavor (if any) for the datastore given as argument.

It should be backward compatible with existing swh datastore modules (i.e.
not yet adapted to this new API).

will be used to implement several test scenarios in following commits.
Also rename existing sql scripts with a "legit" numbering scheme.

To install the new hook:

  $ pre-commit install -t commit-msg

allowing to keep track of state-machine/workflow like status changes as
statsd gauges.

pre-commit is needed externally, not for the actual swh.core tests

allow to configure statsd tags from environment variable (eg. add
the hostname as tag).

This also drops spurious copyright headers to those files if present.

Related to T3812

defined in pytest_plugin. Also move the FakeSocket class there. It
should make it easier to write statsd-related tests in dependent
packages.

Exception thrown when packing a datetime without tzinfo has been changed
from TypeError to ValueError.

It depends on psycopg 3 instead of psycopg2; but we still use psycopg2 everywhere

Some archive files do not have any extension but we can guess their
format by getting their mime types.

So add a last resort processing step to get archive format from mime
type prior trying to extract it.

Related to T3468

Rust crate files are standard tar.gz archives so add support
to uncompress them through shutil.register_unpack_format.

For the record, those kind of source files archive are processed
by the nixguix loader.

By default shutil.unpack_archive will try to match the archive format
from its extension but it does not do it in a case insensitive way.
Thus trying to extract a file named "archive.ZIP" will fail.

So try to detect archive format in a case insensitive way by processing
the data returned by shutil.get_unpack_formats prior unpacking.

Nothing should be printed to standard output when calling
swh.core.tarball.uncompress.

Related to T3468