Antoine Lambert authored 6 months ago and Antoine Lambert committed 6 months ago

The recent addition of the visit_types field in origins endpoint responses
significantly degraded the performance of the HTTP requests as visit types
info are fetched on the backend side by sending requests to elasticsearch
for each origin in a page.

It has been observed by an endpoint client (OpenAIRE) that requests could
fail by returning read timeout errors related to elasticsearch not responding
in time when fetching visit types info.

So revert the addition of that field in order to significantly speedup response
time as requests to elasticsearch will no longer be made by the backend.

Antoine Lambert authored 6 months ago and Antoine Lambert committed 6 months ago

Mercurial branch name for a release has the following pattern:

	tags/{release_name}

Package manager branch name for a release has the following pattern:

	releases/{release_name}

So handle these edge cases when attempting to fetch a release by its
name to avoid 404 errors.

Origin visits having the same date are accessed by visit ids instead
of timestamps but the generated URLs were missing the origin_url query
parameter leading to 404 errors.

Fixes #4809.

Reorder badges URL patterns as URL pattern taking a SWHID as
argument should be checked prior the one taking object_type
and object_id as arguments (as a qualified SWHID can also match
that URL pattern).

Return more meaningful HTTP status code when an error occurs
when attempting to generate a badge.

Generated bearer tokens are stored encrypted in webapp database.

To encrypt a token, the django secret is used plus the user id as salt.

Previously, secret rotation was not taken into account which could result
in token that could no longer be decrypted.

So handle fallback secrets when attempting to decrypt a token and re-encrypt
token with new django secret when secret was rotated.

Also, return a more meaningful error when a secret could not be decrypted.

This way we could finally remove the deprecated swh.loader.git.from_disk module
that would be not used anymore.

We catch MaskedObjectException at the webapp level and display a
specific error page. The page lists the opaque identifier of the masking
requests and the legal email address where to send inquiries.

We also add support for MaskedObjectException in API responses.

Thanks to @anlambert for deduplicating the error templates and fixtures.
Thanks to @vlorentz for improving the wording of the error page.

That make it possible to show the end user the exception for the failure
of the loading task, if any.

Related to #4805

Add /provenance/whereis/ Web API endpoint wrapping call to whereis
operation from swh-provenance API. It enables to get a qualified SWHID
with provenance info from a core SWHID.

Add /provenance/whereare/ Web API endpoint wrapping call to whereare
operation from swh-provenance API. It enables to get qualified SWHIDs
with provenance info from a list of core SWHIDs.

These endpoints are protected by authentication and a user permission
named "swh.web.api.provenance".

Related to swh/infra/sysadm-environment#5397.

It enables to simplify the declaration of a webapp URL containing
a SWHID in its path.

Previously, it was hard to display the most recent counter value in
histogram tooltip. This is no longer the case with this change.

Add Web API endpoint /api/1/origin/save/bulk/requests/ enabling a user
to list the save bulk requests he submitted and get info URLs about them.

Related to #4802.