Antoine Lambert authored 11 months ago and Antoine Lambert committed 11 months ago

Previously, the Permalinks tabs was not really usable with small resolution
display and it was hidden on mobile devices.

To ensure the Permalinks tab can be used on all display sizes, the following
changes were made:

- tab width is dynamically updated according to screen width
- tab anchor position is dynamically updated according to screen height
- tab content is now more compact
- badges and iframe links are hidden on small resolution display
- when tab content cannot fit the screen, it is made scrollable
- tab size is updated when browser window is resized

Related to #4798 and #4799.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 11 months ago

When there is pending save code now requests to be reviewed by staff users,
add a notification icon in the top right corner of the save code now admin
menu entry icon in left sidebar.

Prevent flaky tests on Jenkins.

The X-Original-Forwarded-For header value has the following format:

  client[, proxy1, proxy2]

So ensure to handle all cases when extracting client IP from it to
avoid error when django-ratelimit processes a content view request.

While charset_normalizer performs usually better than chardet, it
remains some edge cases where it can fail to detect content encoding
while chardet succeeds.

So try to detect content encoding with chardet as a fallback when
charset_normalizer failed.

Django >=4 compatibility has been fixed upstream so remove use
of patched django-js-reverse-swh.

Traceback was escaped for HTML views as they are displayed in error page
when debug mode is activated, so ensure to unescape it before displaying
it when the pytest assertion fails.

The charset_normalizer.detect function can return a dict filled with
None values when it failed to detect an encoding so ensure to add a
None check to avoid runtime error when a content encoding cannot be
detected.

It is now only used in the test suites so no need to keep it as
a runtime dependency, also remove not required lxml dependency.

HtmlPrettifyMiddleware uses the `prettify()` method from BeautifulSoup.
Its documentation says:

> Since it adds whitespace (in the form of newlines), `prettify()` changes
> the meaning of an HTML document and should not be used to reformat one.
> The goal of `prettify()` is to help you visually understand the structure
> of the documents you work with.

Having HtmlPrettifyMiddleware enabled therefore leads to surprising
behavior, like extra spaces surrounding “something” when writing
`<code>something</code>` in a template.

Now that Django templates are prettified using `djlint`, this middleware
only has downsides. Let’s remove it entirely.

It has been observed that some users are scraping the browse content
views at a high rate by sending too many requests in a short amount
of time.

So add a rate limit of sixty requests per minute to content views to
avoid such abuse by using the django-ratelimit package.

Related to #4795.

Chardet does not seem really maintained and failed to properly detect some
encodings, resulting in 500 errors when attempting to browse some contents.

So prefer to use the charset-normalizer package instead which is better at
detecting encoding while being faster than chardet.

Paul Wise authored 11 months ago and Antoine Lambert committed 11 months ago

Uses a hidden disabled selected option with an empty value to do this.

Prevents accidentally submitting forges as the first forge type,
which is currently bitbucket, which is rarely used on the web.

The AFN submission API already rejects an empty forge type,
giving a reasonably good error to the end-user.

Using a postcss plugin now results in a lot of displayed warnings so
prefer to use a dedicated webpack plugin to trigger CSS linting.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 11 months ago

Restore previous behavior and select by default the latest visit of
an origin regardless of its type.

This reverts commit 9c2e3a12.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 11 months ago

Align returned data with other endpoints related to origins by adding
visit types info in order to fix errors in the search Web UI.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 11 months ago

This code was used at the time to get extra info (notably stack trace) about
a save code now task but this no longer works since a while and it is redundant
with sentry so better removing it.

Those are no longer rendered after previous commits preventing XSS
so prefer to simply display URLs instead.

This avoids an XSS avenue within errors

swh-storage revision_log() method is expected to return Revision objects
in a near future, so make sure we handle this properly.

It seems we depend on v3 now (v2 generates an AttributeError for
webpack_loader.loaders).

This matches what swh-search returns and expects. This could make
test_lookup_origin() (in test_archive.py) fail when the bad luck (?) made
hypothesis generate a test set where swh-search's in-memory backend
tries to merge the visit_types with an existing one (thus trying to add
a pair set objects).

Adapt the definition of OriginInfo (more or less the local version of
swh-search's OriginDict) accordingly.

Since the release of pytest 8.1, some pytest options are no longer
needed and editable install can be used when running tests using tox.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 1 year ago

When searching for origins with the Web UI, if the search pattern is equal
to the URL of an archived origin then user gets automatically redirected to
the interface for browsing it.

Previously the visit type was not taken into account prior performing the
redirection so ensure to redirect only if the origin has a visit type
that matches the selected one.

Instead of writing a dummy webpack-stats.json file into the static folder
of swh-web when pytest is configured, prefer to autouse a session scoped
fixture mocking the return value of WebpackLoader.load_assets.

Use new option consider_namespace_packages to fix tests execution
with latest pytest release.

Some Jupyter notebooks rendered by the webapp might require the use of
mathjax extensions that were not available in the static assets.

So ensure those extensions are available and can be lazily loaded when
required.

Those are not of interest and generate a lot of noises in swh-web dashboard
from Sentry Web UI.

To support secret key rotation, allow setting SECRET_KEY_FALLBACKS in
the swh-web yaml config.

Introduce support of SECRET_KEY_FALLBACKS for inbound email validation.