Related to swh/meta#4960

Related to swh/meta#4959

This fixes python 3.7 support due to poetry, a dependency of isort, that
removed support for that Python version in a recent release.

In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Also include them the proper sphinx way.

Related to T4496

- pre-commit from 4.1.0 to 4.3.0,
- codespell from 2.2.1 to 2.2.2,
- black from 22.3.0 to 22.10.0 and
- flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies.

Also change flake8's repo config to github (the gitlab mirror
being outdated).

That hook can be frustrating as it can discard a long commit message
if it finds a typo in it so better removing it.

Related to T3922

black is considered stable since release 22.1.0 and the version
we are currently using is quite outdated and not compatible with
click 8.1.0, so it is time to bump it to its latest stable release.

Please note that E501 pycodestyle warning related to line length
is replaced by B950 one from flake8-bugbear as recommended by black.
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Related to T3922

This changes the deprecated :show-nested: instruction to the :nested: one [1].
This also fixes warning about misindented block [2].

[1] https://sphinx-click.readthedocs.io/en/latest/usage/#directive-click

[2]
```
06:22:34  swh fs mount:1:Unexpected indentation.
06:22:34  make: *** [../../swh-docs/Makefile.sphinx:32: sphinx/html] Error 2
```

Due to test modules being copied in subdirectories of the
build directory by setuptools, it makes pytest fail by raising
ImportPathMismatchError exceptions when invoked from root
directory of the module.

So ignore the build folder to discover tests.

To install the new hook:

  $ pre-commit install -t commit-msg

This fixes the warning about the archive.softwareheritage.org link in the markdown.

```
09:45:50  Warning, treated as error:
09:45:50  /var/lib/jenkins/workspace/DFUSE/tests-on-diff/docs/tutorial.md:85:hardcoded link 'https://archive.softwareheritage.org/api/' could be replaced by an extlink (try using ':swh_web:`api/`' instead)

```

Related to D6967

This also drops spurious copyright headers to those files if present.

Related to T3812

The target is kept so it's still usable if needed. It's dropped from the publish build
as this creates problem otherwise.

Related to T3665

Enable to check package documentation can be built without producing
sphinx warnings.

The sphinx environment is designed to be used in continuous integration
in order to prevent breaking documentation build when committing changes.

The sphinx-dev environment is designed to be used inside a full swh
development environment.

Related to T3258

Now that the .status file behavior is consistent, we can safely rely on
it during unit tests.

Close T2830.

Since those entries are updated in the background, we invalidate the
cache on a regular basis.

Close T2864. This should also fix the second problem of T2830, namely
the fact that get_entries(by_date, offset) was not reliable because of
the offset supposing the entries were the same every time.

Since metadata and history already share the same database, it makes
sense to share the connection as well, to prevent unnecessary locking on
the db file.

Related to T2830.

Invalidate specific cached artifact using `rm cache/{SWHID}`

Fix T2889.

this is more consistent with the style used by other SWH modules,
and also avoids parsing errors by our dependency graph generator

Fix T2825.