Related to T4030

Related to T4027

Related to T4026

Related to T3990

This explicits the deposit's type, either 'code' or 'meta'. The uri column is now filled
depending on that type. Mostly, the parsing of the deposit's raw xml content is done to
try and extract that potentially provided information.

If not found, this uses the same heuristics as before (either use the origin_url if
provided or extract the origin out of the swhid_context).

Related to T3680

Closes T3970

It is time to make the guided tour public.

Also remove old help page as its content is now redundant with swh
sphinx documentation. Clicking on the help icon will now launch
the guided tour for all users.

Move workarounds introduced in f7b35626
and 55aa143d from requirements.txt
file to requirements-test.txt as those break debian package installation.

It enables to update origin counters more frequently in the coverage
widget of the production webapp.

Also disable cache use when running swh-web in development or staging
environment to get accurate counters on each homepage refresh.

Add functions to check if we are running a development, staging or
production version of swh-web.

When loading task types are available for these visit types in the
swh deployment environment (currently staging and docker), display
the associated counters in the coverage widget.

Related to T3945

Related to T3835

If one of the URL links to parse contains a comma character, the parsing
was incorrect.

So fix that issue by updating the regexp use to format the links for
display in the API Web UI.

Related to T3967

When calling "./bin/install" from swh-environment in a freshly created
venv, for some reason pip will end up installing flask==1.1.2 when
processing swh-web (likely related to the venv state after installing
other swh modules).

That flask version is quite outdated and will fail to import the json
module from the version of itsdangerous installed in the venv.

So as a workaround pin flask to the first version that does not trigger
that ImportError issue.

markupsafe 2.1.0 removes functions that were used by Jinja2 < 3,
the latter being a dependency of pybadges.

As Jinja2 < 3 is installed in the venv, pinning markupsafe to
a version still compatible with Jinja2 < 3 is the only way to
avoid error when running pytest as some plugins import jinja2.

When attempting to re-encode non UTF-8 textual content, use chardet
to find the encoding first and use it if the detection confidence
is really high.

Previously some encoding like SHIFT_JIS (for japanese language) were
not correctly detected and thus content were badly rendered in the
browse Web UI.

Add support for Bazaar repository loading in Save Code Now.

Simplify test to ensure it does not fail when building debian packages.

Tarballs are no longer loaded as branches but as releases in the
nixguix origin snapshots.

Currently, in case of a high number of visits for an origin, this makes the page
querying the result crash (with an empty cache). Note that even with a populated cache,
the current version could take some time to process.

The following diff opens a function which uses simpler storage api calls to retrieve the
most recent visit from a given point in time (with its most recent associated visit
status) instead of filtering all visits of an origin to retrieve the information. Hence
no longer relying on the cache of all visits for that origin either.

Related to T3905

To install the new hook:

  $ pre-commit install -t commit-msg

Numerous tests are broken since rDSCHc46ffadf7adf24c7eb3ffce062e8ade3818c79cc
has landed in swh-scheduler.

Ensure to restore the old loading task names for tests to succeed.

So we can show this to beta-testers who aren't staff.

This will log all incoming HTTP requests on update events (assuming the user
is authorized), for audit purposes.

Instead of a generic 500.

This will be useful for debugging and checking records are indeed
configured.

Hypothesis can generate multiple times the same origin URL which
makes test fail due to the use of django cache in get_origin_visits.

So force the generation of a single hypothesis example for that test.

There is a lot of origins without any full visit in the archive so in
that case the origin visits page looks empty when reaching it as the
default is to display full visits with different snapshots.

So prefer to display all visits by default and disable the full visits
switch radio buttons when there is no such visits.