To install the new hook:

  $ pre-commit install -t commit-msg

allowing to keep track of state-machine/workflow like status changes as
statsd gauges.

pre-commit is needed externally, not for the actual swh.core tests

allow to configure statsd tags from environment variable (eg. add
the hostname as tag).

This also drops spurious copyright headers to those files if present.

Related to T3812

defined in pytest_plugin. Also move the FakeSocket class there. It
should make it easier to write statsd-related tests in dependent
packages.

Exception thrown when packing a datetime without tzinfo has been changed
from TypeError to ValueError.

It depends on psycopg 3 instead of psycopg2; but we still use psycopg2 everywhere

Some archive files do not have any extension but we can guess their
format by getting their mime types.

So add a last resort processing step to get archive format from mime
type prior trying to extract it.

Related to T3468

Rust crate files are standard tar.gz archives so add support
to uncompress them through shutil.register_unpack_format.

For the record, those kind of source files archive are processed
by the nixguix loader.

By default shutil.unpack_archive will try to match the archive format
from its extension but it does not do it in a case insensitive way.
Thus trying to extract a file named "archive.ZIP" will fail.

So try to detect archive format in a case insensitive way by processing
the data returned by shutil.get_unpack_formats prior unpacking.

Nothing should be printed to standard output when calling
swh.core.tarball.uncompress.

Related to T3468

It's needed to properly read from storage.directory_get_entries,
which returns None instead of empty iterators when a directory
is not found (so clients can tell the difference with the empty
directory without hardcoding its hash).

Do not leak internal swh.core.tarball implementation in tests.

Also fix swh.core.tarball.uncompress documentation and implementation,
the function should not return a value.

Python zipfile module does not support the unpacking of ZIP archive
with legacy compression type 6 (implode).

Fallback using the unzip command in that case as it supports the
unpacking of such ZIP file.

Closes T3369

Use lowercase only for requirement names and sort them in
lexicographical order.

Move types-* in requirements-test.txt files as they are not
required for execution.

Remove no longer used requirements-test-db.txt file and associated
testing-db extra in setup.py.

This ensures mypy will not produce errors while being able to run tests
regardless pytest-postgresql version (<3 or >=3).

Two changes affect us:

* pytest_postgresql.factories.get_config was moved to pytest_postgresql.config.get_config
* all db_name keywords and attributes were renamed to dbname

For consistency, I renamed db_name to dbname everywhere, so this will
affect other SWH packages.

When building the global swh documentation, all commands defined
in other swh packages will be insterted in the swh.core cli
documentation but we only want to document the root command here.

This also fixes the remaining sphinx warning when building the global
swh documentation.

3.0 breaks backwards compatibility, we should handle the upgrade in a
separate patch, for now this ensures the rest of the pipelines can keep
running.

The previous implementation would not normalize permissions for
files detected as executable. This would make the loader crash when a
file has bogus permissions such as 0o100100 (executable but not
readable).

This adds a test that all possible file permissions properly normalize
to either 0o100644 or 0o100755.

Enable to check package documentation can be built without producing
sphinx warnings.

The sphinx environment is designed to be used in continuous integration
in order to prevent breaking documentation build when committing changes.

The sphinx-dev environment is designed to be used inside a full swh
development environment.

Related to T3258

Related to T2265

This is needed for the swh-counters module, one backend uses Redis to
manage the counters by themselves, and another one manages the history
files using prometheus.
Each one can possibly evolve separately or can use a different backend
implementation one day or another.

Related to T3165

I don't understand what this library does or why we needed it.
This commit replaces it with the good old functools.wraps(),
which fixes this crash that started occuring with
decorator 5.0.5:

```
ERROR    testapp:app.py:1892 Exception on /foo [POST]
Traceback (most recent call last):
  File "/home/dev/.local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/dev/.local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/dev/.local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/dev/.local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/dev/.local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/dev/.local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/dev/.local/lib/python3.7/site-packages/decorator.py", line 231, in fun
    args, kw = fix(args, kw, sig)
  File "/home/dev/.local/lib/python3.7/site-packages/decorator.py", line 203, in fix
    ba = sig.bind(*args, **kwargs)
  File "/usr/lib/python3.7/inspect.py", line 3015, in bind
    return args[0]._bind(args[1:], kwargs)
  File "/usr/lib/python3.7/inspect.py", line 2930, in _bind
    raise TypeError(msg) from None
TypeError: missing a required argument: 'self'
```

https://github.com/micheles/decorator/issues/109

There were some typos in the factory's documentation,
and the instances were not documented at all.