Refs. swh/infra/sysadm-environment#4984

Refs. swh/infra/sysadm-environment#4984

Update to upstream version '5.8.0'
with Debian dir c4c13fc02e5eb98ab9e9ba8304789fea2e0253d3

Depending on some instances, we have some specific heuristics, some instances:
- have summary pages which do not not list metadata_url (so some
  computation happens to list git:// origins which are cloneable)
- have summary page which reference metadata_url as a multiple comma separated urls
- lists relative urls of the repository so we need to join it with the main instance url
  to have a complete cloneable origins (or summary page)
- lists "down" http origins (cloning those won't work) so lists those as cloneable https
  ones (when the main url is behind https).

Refs. swh/devel/swh-lister#1800

Update to upstream version '5.7.4'
with Debian dir 82e552590a2590c5d29f97ed55a947ad51ce51d1

This fixes tests hang when building package for debian buster.

Update to upstream version '5.7.3'
with Debian dir 49100f17570e7c3b7a0a7bc5d90835d88b265829

When relisting an opam instance and the opam root directory is already
populated, the '--set-default' parameter must be provided otherwise the
following error is reported:

No switch is currently set. Please use 'opam switch' to set or install a switch

Related to swh/infra/sysadm-environment#4971.

Ensure opam errors are displayed when attempting to list all packages
in order to ease debugging.

Related to swh/infra/sysadm-environment#4971.

Use subprocess.run instead of subprocess.call and subprocess.Popen to
call opam commands and set check parameter to True in order to raise
CalledProcessError exception when an opam command failed.

This should help spotting issues with the opam lister.

Related to swh/infra/sysadm-environment#4971.

Update to upstream version '5.7.2'
with Debian dir e075cf3cf7c1efc8d3a80aaf7e18b5aba7b97d44

In contrary of gitea listing which does not require to provide the q query
parameter, it is required for the gogs case.

After reading the gogs source code, the /repos/search endpoint generates
a sql request of the form: "SELECT * FROM repos WHERE name LIKE '%{q}%'".
By setting the q parameter value to "_", the LIKE clause acts as a
wildcard and all repositories are ensured to be returned.

Fixes #4698.

Update to upstream version '5.7.1'
with Debian dir 56159d6c044097af0681254a772514efc61a6105

Missing docstring prevents the task type to be registered in scheduler
database.

Update to upstream version '5.7.0'
with Debian dir 2531791b0237c82046c8bf50d2afeffe24b8257e

Pagure is a git-centered forge, python based using pygit2.

Its REST API enables to easily list all projects hosted in an
instance so the lister implementation is quite simple.

Related to swh/meta#5043.

The default behavior of subprocess is to pull executables from a
hardcoded list, which doesn't work when opam is installed manually in
the user's home directory.

mypy doesn't catch that multiple uses of
`self.listed_origins[origin_url]` in the same statement should be identical.
Using a temporary local variable for it seems to help.

The files we use weigh 440MB, and there are ~600MB of files we don't use

Update to upstream version '5.6.3'
with Debian dir fe78c649b504911e73b1820a4e4d43621960b679

For the ones coming from a tarball. This matches the change happened in the associated
directory loader.

Refs. swh/infra/sysadm-environment#4906

Update to upstream version '5.6.2'
with Debian dir 88bdac5d662727cbd3c65714293509c90120b3d9

Without this, the loader will fail.

Refs. swh/meta#4979