Those were missed by continous integration because the httpdomain warnings
were disabled on Jenkins (see D6193).

It will simplify the adding of new visit types for Save code now.

Related to D5992

Both Firefox and Chromium replace ':' (with spaces and underscores respectively)

Add new origins types that we now archived.

Update source code provider logos to png only in order to ease the
integration of future ones

Split origins in multiple categories: listed, legacy and deposited.

Add global origins counts per origin type.

Add detailed origins counts for each listed forge, those details are
hidden by default and can be displayed on demand through collapsible
elements.

Remove dead code.

Related to T3127

In order to ease the integration of new visit types for the "Save code now"
service, filter out those with no associated task type in scheduler db.

It means that a new visit type can be added to "Save code now" implementation
but it will not be available until the associated scheduler load task type
gets created. It enables to test the new visit type in dev environments
and deploy safely to production if the associated scheduler load task type
did not get created yet.

Also remove the internal endpoint to get savable visit types, those can
be simply added in Django template.

As that change broke a lot of tests due to scheduler being previously
mocked, most of the tests related to "Save code now" got improved by
removing the mocks and using the swh_scheduler pytest fixture.

Related to D5992

When a content cannot be displayed (for instance due to a too large size),
remove script tag from template to prevent execution errors.

Closes T3454

Only revisions for now. We'll add other object types after
we make sure it works properly.

Using a single example skips a lot of useful tests...

v1.0.0 changed the interface and we need the new one.

It's brittle, and not the right place.

This is to match changes in the vault interface, see D6112 for the rationale.

All existing queries still work, through "fallback" views.  There are
some minor changes in returned fields (removed obj_type, and replace
obj_id with swhid), but I don't think anyone used them (since that info
is already available to them before they send the request).

So tests don't have side-effects

It should be imported from hypothesis.strategies instead from the internal
private module.

This fixes Python tests when building debian buster package.

Running metadata tests with multiple hyothesis examples uncovered those
issues in api-1-raw-extrinsic-metadata-swhid Web API view:

  - RawExtrinsincMetaData only targetting core SWHIds must be provided as test inputs.

  - link-next URL for pagination was invalid.

  - next_page_token must be encoded before providing it to urlsafe_b64encode.

When running swh-web tests using 'make test-full', multiple hypothesis examples
are provided as test inputs instead of a single one when running 'make test'.
In that case some tests were failing mostly due to the fact they were not stateless
between test runs.

That commit fixes those tests execution and ensures stateless test runs by:

  - turning some hypothesis strategies into stateless ones

  - turning the archive_data fixture into a function scope one

  - using subtest fixture from pytest-subtesthack when it is required to reset
    the archive_data fixture for a test between hypothesis example runs

As a consequence, tests will be longer to execute as global state will be reseted
between each test.

Nevertheless, metadata related tests are still failing when running with multiple
hypothesis examples, fix will be handled in next commit.

Related to T1695

Contents with mimetype starting with application/ but with
no detected programming language were not processed for
display but they can be regular text files.

Closes T3486

Useful for the dev environments (a connection to the vpn is needed)

This diff introduces a new config parameter "enable_ql"

If it's set to yes/True, the swh-search query language can be used for the archive search

While using the /origin/search API endpoint, "use_ql=true" can be
passed to use the url_pattern as the search query language input

api_raw_extrinsic_metadata_swhid: Make the docstring consistent with api_raw_extrinsic_metadata_swhid_authorities

Reword some sentences and add step showing SWHID with lines qualifer.

Fix redirection at end of tour when started from homepage.

Related to T3437

D5982 did not fix the flaky cypress test.

Removing cy.intercept hack required some times ago seems
to do the trick.

It will enable to exploit those data in the archive coverage widget
implementation to get deposit counts by origins type.

Related to T3127

Footer is put in cache by Varnish so an old version will be displayed
while the layout template is not updated.

So only display the version in development mode.

Antoine Lambert authored 3 years ago and Antoine Lambert committed 3 years ago

Add a Web UI guided tour implemented using the intro.js library.

The tour showcases the main browsing features offered by Software
Heritage web application by highlighting the relevant DOM elements
and displaying help texts in tooltips.

The tour can span multiple web pages and steps are configured in
a YAML file for help texts writing commodity.

Currently, the homepage, the main origin browsing view and content
browsing view are covered by the tour.

Related to T3202

Antoine Lambert authored 3 years ago and Antoine Lambert committed 3 years ago

This new major version of highlight.js brings notable improvements in
terms of language parsing.

As the github theme we were using has been updated to match the current
github.com one, we now use a custom theme adapted from the old github
one to keep the same look and feel for code highlighting as before.

Closes T3368

The visit is already recursive, we don't need to add this explicit
recursion.

Children nodes can be dispatched to visit_field_name and
visit_field_body instead.