Try to be nice with sysadmins... Also the code in `backends.py` actually
depends on these names...

So that we can get rid of vault task types being created from swh-schedulers'
sql init scripts.

Due to an invalid test and reference path, the master reference was
missing in the git repository cooked from a SWH release.

The following git-fsck error was reported:

	notice: HEAD points to an unborn branch (master)

It might happen that the scheduler RPC service is failing when attempting
to submit a vault cooking task.

So prefer to insert cooking task info in vault database once the task
successfully created by the scheduler. Previously, the non created
cooking task was inserted in database with a null task id and a new
status which prevented to create a new one once the scheduler service
rpc issue fixed.

Related to swh/infra/sysadm-environment#5055.

This fixes the sdist build warnings about importable packages (swh.vault.sql,
swh.vault.sql.upgrades). [1]

This aligns with the swh.storage's setup.py.

[1] for package in swh.vault.sql swh.vault.sql.upgrades
```
    ############################
    # Package would be ignored #
    ############################
    Python recognizes '$package' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    '$package' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that '$package' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.
```

Refs. swh/infra/sysadm-environment#5044

The symlink representation is slightly ambiguous, so not necessarily
very useful.
Additionally, it made directory-flat bundles differ from a non-recursive
'git clone' checkout, which confuses Guix and potentially other tools.

We may revisit this in the future, eg. by adding a dotfile in the
directory or metadata at the root.

This new method returns a direct download URL for a cooked bundle
if the vault cache backend supports the feature.

The backend implementation simply wraps a call to the download_url
method from the objstorage used as vault cache.

Related to #885.

Since commit swh/devel/swh-core@89d48572, the "swh db init" command attempts
to instantiate the postgresql backend of a swh module using the get_datastore
function of the module with a "cls" and "db" parameters.

While it works fine for most of swh modules, it does not for the vault as more
parameters are expected by the "get_vault" function and thus the database init
is failing. The issue was spotted in the docker environment after updating the
swh/stack image.

So extract postgresql backend initialization for the vault in a new class named
VaultBackendDataStore and set it as swh.vault.get_datastore attribute value.

It now needs types-click which is indeed a dependency of
swh.core[testing].

Latest sentry-sdk release now raises a BadDsn exception when
DSN format does not match the expected one.

That function from swh.core.config was removed in swh/devel/swh-core@2663c0a4.

GitLab will display the content of the README file when browsing the
repository. But in case the file is a symlink, it will display the
path pointed by the symlink. There is a 6 year old issue about this:
https://gitlab.com/gitlab-org/gitlab/-/issues/15093

We can workaround the issue by having the content at the root of the
repository and a symlink to this file in the `docs/` directory.

Tested in swh-py-template!27

Related to swh/meta#4960

Related to swh/meta#4959

This fixes python 3.7 support due to poetry, a dependency of isort, that
removed support for that Python version in a recent release.

In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Also include them the proper sphinx way.

Related to T4496

Previous URL was starting with /api/1/vault/git_bare/ leading to a 404
while the correct one must start with /api/1/vault/git-bare/.

- pre-commit from 4.1.0 to 4.3.0,
- codespell from 2.2.1 to 2.2.2,
- black from 22.3.0 to 22.10.0 and
- flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies.

Also change flake8's repo config to github (the gitlab mirror
being outdated).

to more reasonable versions...

instead of creating it in the VaultBackend constructor: when configured
(with host and port), SMTP.connect() is immediately called, which makes
it mandatory to have the smtp server up and running to be able to create
the VaultBackend object (which makes it hard to run properly in an elastic
environment like docker or k8s).

This also removes the fallback to hardcoded 'localhost:25' smtp server;
if the smtp server is not configured or not reachable, the call to
_smtp_send() will fail logging the failure (both in logging and sentry).

Hash dicts are now prefered by swh-objstorage, in order to support
individual hash collisions.

because this later is not yet available on debian, and we do not use new
API features anyway in test_cookers.py

Should fix deb packaging.

'local' has been deprecated for a while.

This is not a runtime issue, because when the supplied mode is DentryPerms.symlink,
the target is only used passed to os.symlink, which does allow str too.

This can happen for various reasons, such as running the vault on
a mirror that is partially replayed

objstorage.add() will change/remove its return value in a future version,
and we do not use it in the vault anyway.

This would be more usefully implemented by generating a (temporary) URL
for direct download of vault bundles from the azure bucket.

This fixes test build [1]

[1] https://jenkins.softwareheritage.org/view/swh%20master%20(draft)/job/DVAU/job/tests/1575/console

Related to T4305