With these changes, GraphQL web client can use Keycloak to login a user
in order to perform authenticated calls to the GraphQL API.

Related to swh-graphql#4652

It enables to test graphql authentication feature in the docker environment.

It is a lightweight service and is getting ready for production so
better moving it in the main docker-compose file.

Also update configuration as there is no keycloak service available
by default in the docker environment and bump max query cost as
it is a testing environment.

Related to swh/infra/sysadm-environment#4803

Use the same mypy version as in tox files for swh packages.

Related to swh/meta#4960

This script does the following:
 - check if the repo is hosted in the SWH GitLab
 - check if the repo is already forked, then forks it
 - if requested, adds Jenkins as Developer
 - if requested, sets up a `forked` remote to the forked repository

Add two new config entries
1: introspection: to enable/disable schema introspection
2: max_query_cost: to reject too big queries

Since the migration to GitLab, merge request branches are now stored in
the git repository of swh-core.

As the total number of branches now exceeds 1000, multiple Web API calls
are now required to get all branches from the snapshot of that repository
loaded into the test archive.

The hierarchy changed recently.

This is using the new swh container gitlab registry and install such binary.
This demonstrates how to use that new container while allowing dev on the svn
loader (using that new binary) to happen.

Install rpm2cpio and cpio packages in the docker image.

Declare task queue in swh-loader service configuration.

flower service has been removed so we can also remove its reverse proxy
configuration.

As in production settings for the objstorage reverse proxy, set HTTP read
timeout to 1h in order to avoid 504 error (Gateway Time-out) when posting
a large amount of contents to the objects storage.

swh-graphql is now using a UvicornWorker. Change entrypoint to
reflect this.
Add the missing search configuration in the graphql config.

The new swh directory loader will check recursive nar checksum
for downloaded tarball content using the nix-store command if
it is available in execution environment.

Add task queues declaration for following listers: conda, cpan, fedora,
hackage, puppet, rubygems.

Add task queue declaration for following loaders: cpan, hackage, opam,
puppet, rubygems.

Ensure lister task queues are sorted in lexicograhical order.

In order to have a search behavior close to the production one, add services
related to archive search in the default docker-compose.yml file:

  - swh-search: remote search service using a memory backend

  - swh-search-journal-client-objects: journal client feeding swh-search with
    objects loaded into the archive (typically origins)

  - swh-search-journal-client-indexed: journal client feeding swh-search with
    results of swh-indexer processing on archived objects

As a consequence, it simplifies the docker-compose.search.yml file enabling
to use a swh-search service with a more costly elasticsearch backend.

It enables to quickly schedule the loading of listed origins in the
docker environment without having to restart the service dedicated
to this task (swh-scheduler-schedule-recurrent).

Docker now officially supports Docker Compose in the form of a plugin
for the `docker` command.
See: https://www.docker.com/blog/announcing-compose-v2-general-availability/

Following the official Docker documentation for Linux will install the
Docker Compose plugin. We can remove the mention of the `docker-compose`
Debian package.

Command lines and pointers to the documentation are now updated
accordingly.

It enables to update the metrics regarding the number of loaded
origins into the archive.

Those are consumed by the archive coverage widget in the webapp
homepage.

It enables to test those in the docker environment.

Recent versions of pip can now properly install all swh packages
in editable mode plus their dependencies in a single command.

Previously a pip install command was issued for each swh package.

pip 21.3 added support to editable installs for projects that have a
pyproject.toml and use a build backend that supports PEP 660, but mypy
does not detect yet such editable installs and following errors are
reported when type checking swh modules:

  Cannot find implementation or library stub for module named swh...

So temporarily pin pip until mypy properly supports PEP 660 editable
installs.

seems a better idea than sticking to the default behavior (using
the directory name, hence 'docker'), and may even fix the
"network has active endpoints" error on 'compose down'.

make it yield the docker_host on which a check_compose_output lambda has
been added; this allows users of the docker_compose fixture to use it
directly to run `compose` commands in addition to usual commands without
having to explicitly use the compose_cmd fixture.

instead of the old 'docker-compose' tool.

See https://docs.docker.com/compose/#compose-v2-and-the-new-docker-compose-command

mostly for the sake of consistency.

They are now redundant with the pytest-testinfra based ones.

Towards removing the shell based tests for the docker environment,
synchronize the features they cover by checking in Python based test
that all git objects have been correctly loaded into the archive.

Also prefer to use dulwich to manipulate a git repository instead
of git command.

There is no swh directory in swh-environment repository so
the mypy check was failing when committing changes to a
Python test file.