Add Web API endpoint /origin/save/bulk/request/(request_id)/ to get
feedback about a previously sent save bulk request.

The endpoint requires authentication and special permission to be
queried, also only the user that submitted the request or SWH staff
members can query it.

It returns for each submitted origin the following info:

- the origin URL
- the visit type
- the origin status regarding its processing by the bulk save lister,
  either pending, accepted or rejected
- the last scheduling date if any
- the last visit date if any
- the last visit status if any
- the last snapshot SWHID if any
- the rejection reason if the origin was rejected by the lister
- the browse URL if the origin was successfully visited

These info are returned in a paginated way as the number of submitted
origins can be large.

Related to #4802.

Allow to define endpoint route using a simple string instead of a
regular expression.

It notably enables to use django parameter type annotations like
int or uuid.

test_get_snapshot_context_with_origin sometimes fails due to
a date comparison issue (seconds might be offset by 1).

This command will copy all the active mailmaps in the masking proxy DB
which dsn is given as argument.

It is similar to the (now deprecated) sync_mailmaps command, but since
it does not modify the Person table of the storage any more, it does not
need the timestamp-based detection to determine which entries are to be
enabled or disabled; it simply rewrite the whole displayname table
every time it's called, if need be.

A '--force' option allows to enforce rewrite the whole table even if no
update has been detected in the displayname database.

Extract tests for (soon to be legacy) sync_mailmap related fatures in a
dedicated test file.

If not provided, .get() would return None, which makes get_save_origin_requests_to_update
return every SCN request. There is no reason to want the webhook to do that.

When a user request the cooking of a bundle not referenced in the
browser local storage and that bundle has already been cooked but
no longer available in vault cache, ensure the recooking request
is sent as previously that edge case was not handled.

Related to #4804.

test_snapshot_browse_with_id_origin_and_timestamp is flaky due to
a date comparison issue (seconds might be offset by 1) so mark it
as such with pytest.

Apply suggestion from rboyer.

Add javascript function based on the use of the anchorjs package enabling
to add anchors to headings. It enables users to easily get links to parts
of a page by hovering mouse cursor on page headings.

Use it in homepage, api pages but also in pages rendering a README file
from the archive.

Upgrade highlightjs to its latest release and add a couple of new
referenced plugins adding support for new programming languages.

Add a new django application to manage bulk archival of origins.

As a first step, add a new Web API endpoint to submit a list of origins
to archive and a new endpoint enabling to retrieve that list of origins
in a paginated way.

The Web API endpoint enables an user with specific permission to submit
a list of origin URLs and their visit types through a POST request.
The endpoint performs some basic checks on the received origins data to
verify origin URLs are well formed but also if provided visit types are
supported.

If provided origins data are valid, the request is accepted and a oneshot
scheduler task is created to execute the bulk-save lister. That lister will
consume that list of origins, perform some extra but more costly checks on
them and those validated by it will then be scheduled for loading into the
archive.

If some origins data are not valid, the request is rejected and a list of
bogus origins along the reasons of their rejections is returned to the user
in the Web API response.

The other endpoint is dedicated to be consumed by the bulk-save lister to
retrieve the submitted origins list in a paginated way. It is better to
store submitted origins data on the webapp side to avoid bloating the
scheduler database with large JSON documents in case a big list of origins
is submitted by an user.

Related to #4802.

Synchronize it with the one used to render code elements in all pages
of the webapp as it enables to better highlight them.

Antoine Lambert authored 9 months ago and Antoine Lambert committed 9 months ago

This should help avoiding implementation mistakes as mypy will now
check typing when a client code use these interfaces.

Fix some typing issues that were uncovered by this change.

Instead of defining our own django assertion wrappers, prefer to
use those from pytest-django.

OptimizeCSSAssetsPlugin is no longer maintained and must be replaced by
CssMinimizerPlugin.

It implied the migration of the eslint configuration files to the
new flat format (thanks to the eslint-migrate-config CLI tool).