Improve the main template to use better rules to show the
"Administration" section of the left menu.

Add a "Users & groups" entry if the swh.web.admin app is enabled. This
later opens the users admin page (either keycloak or the django admin
page) in a new tab.

Set hypothesis max_examples to 1 and remove subtest fixture use.

It fixes issues with autodoc processing as this django application is
disabled by default.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 10 months ago

Enable to use django administration site in swh-web by including "swh.web.admin"
application in swh_extra_django_apps list from swh-web configuration.

When enabled, the admin site is available through the new /manage/ endpoint.

It allows an user to modify some django models of interest (adding, deleting or
modifying rows) if he/she has adequate permissions to do so.

Currently, only the add forge now request and save code now request models can
be managed with the admin site through the use of the following permissions:

- swh_web_add_forge_now.add_request
- swh_web_add_forge_now.change_request
- swh_web_add_forge_now.delete_request
- swh_web_add_forge_now.view_request
- swh_web_save_code_now.add_saveoriginrequest
- swh_web_save_code_now.change_saveoriginrequest
- swh_web_save_code_now.delete_saveoriginrequest
- swh_web_save_code_now.view_saveoriginrequest

When swh-web is using the default django authentication backend, the admin site
also enables and administrator to manage user accounts and their permissions.

When swh-web is using Keycloak as the authentication backend, the permissions
listed above must be added as swh-web client roles and these roles can then be
set for an user through the Keycloak admin console in order to use the features
offered by the admin site.

It messes up with django permissions creation when using pytest-xdist.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 10 months ago

Add a new migration file for the swh.web.auth django application enabling to
persist webapp special permissions to database in order to be able to use them
when the webapp relies of the default django authentication backend for login
operations (mirror version of swh-web for instance that does not use Keycloak
for authentication).

The permissions can then be set for a particular user using the django admin
views by a django user having the adequate permissions to modify user accounts.

Some tests were modified as the reset_sequences parameter of django_db pytest
mark cannot be used as it breaks persisted permissions.

Python uses symlinks for editable installs which was messing up with the
code adding contents in the test archive from tests resources folder.

It should fix a flaky test on Jenkins.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 11 months ago

Previously, the Permalinks tabs was not really usable with small resolution
display and it was hidden on mobile devices.

To ensure the Permalinks tab can be used on all display sizes, the following
changes were made:

- tab width is dynamically updated according to screen width
- tab anchor position is dynamically updated according to screen height
- tab content is now more compact
- badges and iframe links are hidden on small resolution display
- when tab content cannot fit the screen, it is made scrollable
- tab size is updated when browser window is resized

Related to #4798 and #4799.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 11 months ago

When there is pending save code now requests to be reviewed by staff users,
add a notification icon in the top right corner of the save code now admin
menu entry icon in left sidebar.

Prevent flaky tests on Jenkins.

The X-Original-Forwarded-For header value has the following format:

  client[, proxy1, proxy2]

So ensure to handle all cases when extracting client IP from it to
avoid error when django-ratelimit processes a content view request.

While charset_normalizer performs usually better than chardet, it
remains some edge cases where it can fail to detect content encoding
while chardet succeeds.

So try to detect content encoding with chardet as a fallback when
charset_normalizer failed.

Django >=4 compatibility has been fixed upstream so remove use
of patched django-js-reverse-swh.

Traceback was escaped for HTML views as they are displayed in error page
when debug mode is activated, so ensure to unescape it before displaying
it when the pytest assertion fails.

The charset_normalizer.detect function can return a dict filled with
None values when it failed to detect an encoding so ensure to add a
None check to avoid runtime error when a content encoding cannot be
detected.

It is now only used in the test suites so no need to keep it as
a runtime dependency, also remove not required lxml dependency.

HtmlPrettifyMiddleware uses the `prettify()` method from BeautifulSoup.
Its documentation says:

> Since it adds whitespace (in the form of newlines), `prettify()` changes
> the meaning of an HTML document and should not be used to reformat one.
> The goal of `prettify()` is to help you visually understand the structure
> of the documents you work with.

Having HtmlPrettifyMiddleware enabled therefore leads to surprising
behavior, like extra spaces surrounding “something” when writing
`<code>something</code>` in a template.

Now that Django templates are prettified using `djlint`, this middleware
only has downsides. Let’s remove it entirely.

It has been observed that some users are scraping the browse content
views at a high rate by sending too many requests in a short amount
of time.

So add a rate limit of sixty requests per minute to content views to
avoid such abuse by using the django-ratelimit package.

Related to #4795.

Chardet does not seem really maintained and failed to properly detect some
encodings, resulting in 500 errors when attempting to browse some contents.

So prefer to use the charset-normalizer package instead which is better at
detecting encoding while being faster than chardet.

Paul Wise authored 11 months ago and Antoine Lambert committed 11 months ago

Uses a hidden disabled selected option with an empty value to do this.

Prevents accidentally submitting forges as the first forge type,
which is currently bitbucket, which is rarely used on the web.

The AFN submission API already rejects an empty forge type,
giving a reasonably good error to the end-user.

Using a postcss plugin now results in a lot of displayed warnings so
prefer to use a dedicated webpack plugin to trigger CSS linting.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 11 months ago

Restore previous behavior and select by default the latest visit of
an origin regardless of its type.

This reverts commit 9c2e3a12.

Antoine Lambert authored 1 year ago and Antoine Lambert committed 11 months ago

Align returned data with other endpoints related to origins by adding
visit types info in order to fix errors in the search Web UI.

Antoine Lambert authored 11 months ago and Antoine Lambert committed 11 months ago

This code was used at the time to get extra info (notably stack trace) about
a save code now task but this no longer works since a while and it is redundant
with sentry so better removing it.

Those are no longer rendered after previous commits preventing XSS
so prefer to simply display URLs instead.

This avoids an XSS avenue within errors

swh-storage revision_log() method is expected to return Revision objects
in a near future, so make sure we handle this properly.