This fixes an import error during sphinx-apidoc processing.

It's needed by the ci to be able to reference the api links

Related to T3307

The first two redirect, the last one is broken.

As it was, it was unclear that it linked to the documentation of internal
Python module, instead of the public HTTP API.

Phabricator short code is used as folder name instead of project name
in that case.

Related to T3258

Enable to build the local documentation of each swh package using a
dedicated tox environment.

In order to check for possibly introduced sphinx warnings in the
documentation sources, the documentation is built with the -W option
of sphinx-build.

As some references located in scopes external to the processed
package would not be resolved, related sphinx warnings are
suppressed to allow the doc build to succeed anyway.

Besides the errors related to references that can only be checked
when building the full documentation, all other sphinx errors
should be caught.

Related to T3258

For some reason, neither was included, even though 'autoclass_content'
should default to 'class'.
Anyway, there is no harm in including both.

Software Heritage uses Django in several packages: swh-auth, swh-deposit
and swh-web.

When building global or local package documentation, the environment
variable DJANGO_SETTINGS_MODULE must be set to the adequate settings
module in order to avoid errors with sphinx autodoc processing.

Previous approach was merging swh-deposit and swh-web django settings
but it did not prevent all autodoc errors due to some missing or
unexpected django setting value.

New approach force reset Django settings according to the next rst
document to read by sphinx. This ensures adequate django settings
will be set for each swh packages before processing apidoc generated
rst files.

It also enables to not depend on swh-deposit and swh-web for building
a local package documentation.

Related to T3258

It enables to not depend on swh-web to build standalone swh package
documentation, plus these sphinx features could be reused in other
package documentation.

Related to T3258

- add the scheduler journal client
- remove the indexer journal client (for clarity)
- remove indexer workers (for clarity)
- add a postgresql db attached to the web app

Closes T3229.

It seems to be misunderstood by most people when they start out.

It is now required for swh-web frontend development.

And add environment variable SPHINXOPTCOLOR to disable it on Jenkins

The command is not always installed by default, for instance in the
debian:buster docker image.

These django settings contains the deposit ones but some swh-web settings
are also needed to avoid errors when running autodoc.

Closes T3248

Installation is implicit when using tox but not in a custom make script.

The new pip resolver ends up backtracking numerous packages when
installing all depencies in a single pip command but this is
terribly slow and long to execute.

A workaround is to install each swh package one at a time by
executing pip multiple times, no backtracking happens in that
case and it reduces considerably the overall installation time.

Another advantage of proceeding like this is that it ensures each
swh package will be installed in development mode.

Also increase tox verbosity when calling make to have some feeback
of what it is doing.

It used to only document the core components.

This commit does the following:

* expands the existing descriptions
* reorders them in didactical order
* and all the other components in two new tiers ("major" and "extra", in
  addition to "core")

This extra requirement has been dropped from swh-storage.

I think it's a little clearer this way what Keycloak is
and when a reader should care (or not) about it.

To match the toctree.

I'm leaving keycloak out of it, because it does not actually describe
the architecture. It's only in the architecture toctree for lack of
a better place for now.

This only includes redirects for user documentation;
external links to the developer documentation would be broken,
but I don't think there are any such link and I don't want to
clutter the file tree unnecessarily.

So the docker documentation is available on docs.softwareheritage.org instead
of linking to the source on Phabricator

This adds the roadmap entry in the site in the:
- index menu on the left (prior to the components listing)
- main page under a ` Roadmap` section just before the components section

Related to T3208

Dependencies must be written in topological order, or they will be fetched
from PyPI instead of the local system