This fixes debian package builds of swh-web.

Cypress >= 13.3 blocked the use of sorry-cypress and cypress-cloud in the
cypress binary so we can no longer use these tools to execute parallel
cypress tests on Jenkins.

Fortunately, it exists another tool called cypress-split that can be used
for the same purpose without relying on a tests orchestration server.

So replace the use of cypress-cloud by cypress-split to still benefit of
parallel cypress tests in CI jobs.

Instead of disabling scheduler service when running cypress tests, use
the recently added scheduler backend spawning a temporary postgresql
database.

In order to update the statuses of save code now requests in a push manner
instead of the current pull one, add a new HTTP endpoint to receive webhook
messages about origins being visited by the SWH archive.

When the visit of an origin results from a Save Code Now request, the status
of the request is then updated when receiving webhook messages notifying
the new status of the visit.

Related to swh/meta#4836.

Paul Wise authored 1 year ago and vlorentz committed 1 year ago

There could be non-technical issues with forge archiving, for example
some admins have responded saying they need to obtain export approval.

Fixes: commit 16ba15e5

It enables to cook a git repository for all supported SWH object types:
directory, release, revision and snapshot.

Previously only a revision could be cooked as a git repository through
the Web API.

Some vault cache backends (azure one for instance) can provide a direct
download link for a cooked bundle. In that case, redirect requests to
the /api/1/vault/(bundle_type)/(swhid)/raw/ to such links in order to
offer efficient downloads and avoid connection errors when dealing with
large bundles.

Related to swh/devel/swh-vault#885.

Latest django-stubs release is not compatible with mypy 1.0.1.

Some HTTP clients (java one for instance) can turn double slashes in URL
path into a single one which breaks the use of SWH Web API when querying
the save origin endpoint.

So ensure to demangle such origin URLs received as endpoint arguments to
workaround that issue.

Related to #4769.

Some HTTP clients (java one for instance) can turn double slashes in URL
path into a single one which breaks the use of SWH Web API when querying
origin related endpoints.

So ensure to demangle such origin URLs received as endpoint arguments to
workaround those issues.

Related to #4769.

As multiple snapshot branches can target the same release, multiple
occurrences of the same release could be found in the snapshot content
dropdown located in the top left part of the browse Web UI.

So ensure to remove duplicates by using the ifchanged template tag in
related django template.

As a consequence snapshot releases are now sorted by descending order of
their names and are displayed in the same order in the different parts of
the Web UI.

Currently in staging, the service is failing [1].

Patching it with this, the routine went back to normal [2]

[1]
```
Oct 04 09:40:32 webapp django-admin[2174716]: /etc/softwareheritage/web/web does not exist, using /etc/softwareheritage/web/web.yml instead
Oct 04 09:40:33 webapp django-admin[2174716]: Traceback (most recent call last):
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/bin/django-admin", line 5, in <module>
Oct 04 09:40:33 webapp django-admin[2174716]:     management.execute_from_command_line()
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
Oct 04 09:40:33 webapp django-admin[2174716]:     utility.execute()
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/django/core/management/__init__.py", line 375, in execute
Oct 04 09:40:33 webapp django-admin[2174716]:     self.fetch_command(subcommand).run_from_argv(self.argv)
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 323, in run_from_argv
Oct 04 09:40:33 webapp django-admin[2174716]:     self.execute(*args, **cmd_options)
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/django/core/management/base.py", line 364, in execute
Oct 04 09:40:33 webapp django-admin[2174716]:     output = self.handle(*args, **options)
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/management/commands/refresh_savecodenow_statuses.py", line 27, in handle
Oct 04 09:40:33 webapp django-admin[2174716]:     refreshed_statuses = refresh_save_origin_request_statuses()
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/origin_save.py", line 715, in refresh_save_origin_request_statuses
Oct 04 09:40:33 webapp django-admin[2174716]:     if save_requests.count() > 0
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/origin_save.py", line 683, in update_save_origin_requests_from_queryset
Oct 04 09:40:33 webapp django-admin[2174716]:     task_runs.get(sor.loading_task_id),
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/origin_save.py", line 380, in _update_save_request_info
Oct 04 09:40:33 webapp django-admin[2174716]:     save_request
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/origin_save.py", line 318, in _check_visit_update_status
Oct 04 09:40:33 webapp django-admin[2174716]:     visit_date, visit_status, _ = _get_visit_info_for_save_request(save_request)
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/save_code_now/origin_save.py", line 297, in _get_visit_info_for_save_request
Oct 04 09:40:33 webapp django-admin[2174716]:     ovs = archive.origin_visit_find_by_date(origin, save_request.request_date)
Oct 04 09:40:33 webapp django-admin[2174716]:   File "/usr/lib/python3/dist-packages/swh/web/utils/archive.py", line 1110, in origin_visit_find_by_date
Oct 04 09:40:33 webapp django-admin[2174716]:     return converters.from_origin_visit({**visit_status.to_dict(), "type": visit.type})
Oct 04 09:40:33 webapp django-admin[2174716]: AttributeError: 'NoneType' object has no attribute 'to_dict'
Oct 04 09:40:33 webapp systemd[1]: swh-webapp-update-savecodenow-statuses.service: Main process exited, code=exited, status=1/FAILURE
```

[2]
```
Oct 04 09:45:31 webapp django-admin[2175548]: /etc/softwareheritage/web/web does not exist, using /etc/softwareheritage/web/web.yml instead
Oct 04 09:45:33 webapp django-admin[2175548]: Successfully updated 10 save request(s).
Oct 04 09:45:33 webapp systemd[1]: swh-webapp-update-savecodenow-statuses.service: Succeeded.
```

CRAN lister name has recently been renamed to cran so ensure displayed
metrics are the correct ones.

Related to swh/meta#5053.

Related to swh/meta#5051.

cypress-cloud is currently not compatible with cypress 13.x so
pin cypress version until this is sorted out.

First loadings of source packages from Red Hat based distributions
were made on staging environment so it is time to include the rpm
origin type in the archive coverage widget.

Related to swh/meta#5011 and swh/meta#5042.

When browsing a content by its sha1_git identifier, the identifier of the
directory containing it is prepended to the path query parameter in order
to display a functional breadcrumbs enabling to navigate back and forth
in a filsystem.

However, when an origin context is provided through the origin_url query
parameter of the content view, the previous implementation was using the
root directory identifier of the head branch of the latest origin snapshot
in the breadcrumbs instead of the one provided in the path query parameter.
This could lead to a 404 error as the wrong root directory was used to
query subpaths in the breadcrumbs.

User requests filter was still taken into account otherwise.

This fixes debian package build for swh-web on unstable.

Previously, only user requests was displayed as the checkbox activating
this filter was checked by default.

It enables to enlarge the clickable area of each directory entry link
and thus ease navigation when the entry name contains a few characters.

Related to #4756.

Previously the error responses generated by django custom handlers for
common HTTP error codes (like 404 for an invalid URL for instance) were
always in HTML format and thus the Accept header of HTTP requests were
not honored.

This was pretty confusing when querying a Web API endpoint as HTML could
be returned when an error occured instead of JSON or YAML.

So generate error response with proper content type based on the Accept
HTTP request header value.

Related to #4758.

Refs. swh/meta#5045

Refs. swh/meta#5048