Previously, functions from the archive module to query origin visit info
were returning the latest visit status date instead of the visit date.

This could lead to wrong returned visit when searching a visit by date
through links offered by the webapp as the wrong visit date was used.

Previously, the page size was not taken into account and the default limit
value of 1000 was used when calling get_listed_origins from scheduler API
leading to incomplete API responses.

Fixes #4819.

Dates returned by origin_visit_stats_get from scheduler API are
converted to local timezone which can make tests failed if local
timezone is different from UTC.

This should have been committed in f571c557 but the change was lost
somehow. Citations tab was always displayed while it must be only
when a citation metadata file is available in the browsing context.

Antoine Lambert authored 5 months ago and Antoine Lambert committed 4 months ago

Users having the swh.ambassador permission will have the citations UI
displayed when logged in.

Hypothesis may generate the same URL twice (or more) in the same test
session, in which case the given URL would already be in
archive._origin_info() cache, defeating the test.

So clear django cache completely before running this test.

Antoine Lambert authored 5 months ago and Antoine Lambert committed 4 months ago

Do not show Citations tab by default and activate its display only when
a software citation can be generated from the browse context.

Related to #4815.

Antoine Lambert authored 5 months ago and Antoine Lambert committed 4 months ago

When a parsing error occurs when attempting to generate a citation,
store error message in Citation dictionary and return an error response
enriched with SWHID of citation metadata source file.

It enables to get feedback to users about errors in the citations Web UI
while offering link to the bogus metadata file.

Antoine Lambert authored 5 months ago and Antoine Lambert committed 4 months ago

Some deposit XML metadata are provided non formatted on a single line
so ensure to prettify them before display in the deposits admin Web UI.

Using a CharField with at most 200 characters was too restrictive
as some URLs can exceed that length so bump the maximum number of
characters to 4096.

Resolves #4818.

Using a CharField with at most 200 characters was too restrictive
as some URLs can exceed that length so bump the maximum number of
characters to 4096.

Drop the mention of the request since it can cause admin concerns.

Switch from the "forge" terminology to "code" and site" since
"forge" is outdated terminology since the advent of GitHub,
which did not keep up the tradition of old code forges.

Mention that the archiving will be regular too.

Antoine Lambert authored 5 months ago and Antoine Lambert committed 5 months ago

Resolves #4817.

Default configuration uses a remote scheduler backend targeting a non
running server on localhost:2008, but as scheduler rpc client uses a
retry mechanism (recently introduced in swh-scheduler v2.6.2) this
significantly slow down tests as any connection attempt to the non
running scheduler rpc server is retried five times with a 10 seconds
delay.

Some generated citations can be quite long and explode the citation Web UI
height so set a maximum height to each citation pre instead in order to make
them scrollable.

When sending origins in CSV format to the save bulk endpoint, the
text/csv content type must be explicitly specified by the HTTP client.

If a user submitted list of origins to archive contains duplicates
ensure to remove them to avoid error when inserting to database.

Aside enabling that technology preview through configuration to ease
deployment, also enable it by default for staff users.

It enables a staff user to demonstrate that new feature in production
webapp that contains much more repositories with available citation
metadata.

Related to product-management/swh-archive-website#4.

This reverts commit d273ec77.

Previous implementation prevented inserting highlightjs code
in webapp bundle and used dedicated highlightjs bundle instead
so restore it to reduce webapp bundle size.

The max-height value for the sticky sidebar must be computed from the
viewport and top bar heights to ensure proper display on all screens.

Integrate Web UI to generate software citations in BibTex format
in the new Citations side tab on the right.

The Web UI is similar to the one for Permalinks with the difference
that BibTex citations are generated on the fly through Web API calls
when selecting an object type.

This new Citations Web UI and its sidetab are not integrated by
default and can be activated through configuration by setting the
activate_citations_ui entry to True.

Related to product-management/swh-archive-website#4.

Add new boolean config flag activate_citations_ui (default to False)
to integrate or not the Citations Web UI in the browse interface.

In order to get an easy access to the incoming software citations
Web UI for the currently browsed software objects, add a new side
tab whose handle is located below the Permalinks tab one.

When clicking on any tab handles, both tabs are slided to the left
and the current displayed one can be switched by clicking on its
handle once the tabs opened.

Also update guided tour to focus on the citations tab after the
permalinks one, it should be completed when the citations Web UI
will be integrated.

Related to product-management/swh-archive-website#4.

Citation endpoints now return a JSON object containing:

- a formatted citation string in BibTex format

- the qualified SWHID of the source for intrinsic citation metadata
  (citation.cff or codemeta.json file)

- the URL to fetch the raw bytes of the intrinsic citation metadata

Those functions are quite dedicated to the new citation feature so
rename them in order to distinguish them from the ones fetching
various raw intrinsic metadata.

Also improve their documentation a bit.

Antoine Lambert authored 6 months ago and Antoine Lambert committed 6 months ago

Fix endpoints responses documentation.

Wrap long example URLs in variables.

Fix an accessibility issue spotted by cypress-accessibility-checker
version 3.1.76.

The bootstrap admin theme we are using (AdminLTE) recently added support for
bootstrap 5.x in its 4.0 release (still in beta but works for our needs).

So it was time to upgrade bootstrap from 4.x to 5.x for the SWH webapp.

The following was made to complete the upgrade while keeping same look and
feel as before:

- remove use of bootstrap loader for webpack and simplify webpack configuration

- replace use of deprecated node-sass package by sass one from Dart

- update bootstrap sass variables to override and add button style matching the
  default one from AdminLTE 3.x we were using

- adapt some CSS rules and add new ones to ensure same look and feel as before

- remove explicit use of custom form controls from bootstrap 4.x as they are now
  the standard in bootstrap 5.x through new CSS classes form-*

- update CSS classes renamed in bootstrap 5.x and adminlte 4.x, notably:

  * wrapper => app-wrapper
  * main-header => app-header
  * content => app-content
  * sidebar => app-sidebar
  * footer => app-footer
  * ml-* => ms-*
  * mr-* => me-*
  * pl-* => ps-*
  * pr-* => pe-*
  * float-left => float-end
  * float-right => float-start
  * text-left => text-start
  * text-right => text-end

- rename some data-* attributes to data-bs-*

- use btn-secondary CSS class for button style instead of btn-default

- use standard bootstrap close button for modals and alerts

- put sidebar HTML code in its own dedicated include file

- simplify forms HTML code and use new form-label CSS class

- remove no longer needed javascript hacks

- improve color theme consistency notably when focusing elements with keyboard

This private function wraps a call to origin_get function from
swh-search interface and is called for each browse page of the
webapp having an origin context.

Caching its results enables to avoid sending too many requests
to elasticsearch in production.

Fixes #4812.

Antoine Lambert authored 7 months ago and Antoine Lambert committed 7 months ago

The recent addition of the visit_types field in origins endpoint responses
significantly degraded the performance of the HTTP requests as visit types
info are fetched on the backend side by sending requests to elasticsearch
for each origin in a page.

It has been observed by an endpoint client (OpenAIRE) that requests could
fail by returning read timeout errors related to elasticsearch not responding
in time when fetching visit types info.

So revert the addition of that field in order to significantly speedup response
time as requests to elasticsearch will no longer be made by the backend.

Antoine Lambert authored 7 months ago and Antoine Lambert committed 7 months ago

Mercurial branch name for a release has the following pattern:

	tags/{release_name}

Package manager branch name for a release has the following pattern:

	releases/{release_name}

So handle these edge cases when attempting to fetch a release by its
name to avoid 404 errors.