This naming is more explicit about what this function is doing.

This function is used in multiple package loaders so add a throttling
retry policy and debug log about fetched URL.

Some package loaders might encounter errors while attempting to get
package info for a given version (HTTP error for instance).

So handle that case and ensure partial visit when such an error occurs.

Related to T4124

Those methods were not reimplemented and could return incorrect statuses
once the load method called.

As they are useful for tests, implement them in PackageLoader class.

Such test was missing so add a couple of tiny tarballs in tests
data directory and simulate a successful visit for origin example.

Also update origin URL and remove a couple of hardcoded litterals.

Some go packages only have a development version not listed by the
@v/list endpoint but returned by the @latest endpoint.

So ensure to return it in get_versions method or it will be missed
by the loader.

Related to T4124

When a go package name contains uppercase characters in it, associated
goproxy URLs need to be case-encoded by replacing every uppercase letter
with an exclamation mark followed by the corresponding lower-case letter.

This fixes the loading of such packages.

See https://go.dev/ref/mod#goproxy-protocol.

Related to T4124

Make use of packaging.versions instead of distutils versions classes
that are deprecated

Resolves T4528
Related T4465

It enables to search sentry events related to a particular loaded origin
URL or for a given visit type from the sentry Web UI.

Origin URL for a package listed by the pubdev lister is now in the form
https://pub.dev/packages/<package_name> so we need to reconstruct the
API URL to get package versions from it.

Some HTTP download requests might be throttled by remote servers
so add retry mechanism with exponential backoff to fix tarball
downloads in some loaders.

This uses the Golang proxy since that's what `go get` uses, and since it
probably offer better performance than most direct sources.

Fix an issue where get_versions failed when version names does not
follow semver conventions.
Add related test.

The method name was wrong replace it with 'load_pubdev'

Call an http api endpoint per package to get all its versions and
artifacts metadata.

Adapt code, test fixture and tests after the lister split its artifacts
data to artifacts and arch_metadata (see D8259)

Related T4233

Add 'aur' package loader

It seems that despite setting it in the 'except BaseException' block,
it is still occasionally undefined in the 'finally' block when
triggered by a SystemExit exception.

This should hopefully prevent UnboundLocalError from
being raised from the 'finally' block from now on

This fix an issue discovered while testing loader on docker environment,
error was (load_arch() got an unexpected keyword argument
'lister_name')

Related T4233

named args

This fix an issue discovered while testing loader on docker environment,
error was (load_crates() got an unexpected keyword argument 'lister_name')

Related T4104

They should use the same convention as Git.

Fetch Arch linux packages from lister discovered origins.
For each origin it get versions from extra_loader_arguments['artifacts']

Arch Linux package can comes as .xz or .zst file archive.
Support for .zst (Zstandard compression) has been requested with D7993.

Related to T4233

An artifact without time info can be provided in the artifacts list
parameter of the loader (for instance last modification date
is not available for tarballs coming from github releases).

That case was not handled by the archive loader wich was resulting
in loading error so add fix for it.

It will be used by the git loader to run a graph traversal on
received packfiles; and we want to monitor its runtime as a separate
metric.

It will be used by the git loader to compute metrics on the number
of objects ignored by the filtering proxy

The post_load method of a loader can raise an exception so we must
ensure to turn back the success variable to False in that case.

For instance, the subversion loader post_load checks that latest
exported revision is consistent with what the official subversion
client produces. If it is not an exception will be raised to set
the visit status to partial.

Related to T4236

The loader get 'artifacts' information through lister
'extra_loader_arguments'.

Related to T4104

artifacts list sent by the lister is not guaranteed to be sorted
by artifact time so the last element is not necessarily the most
recent artifact.

So ensure to use the most recent artifact as default version by
sorting the artifacts by their ISO 8601 date representation
(designed for lexicographical sort). If a date failed to be
parsed by the lister (edge case but can happen), the time value
will be an empty string so the sort operation will not end up
with error.

Related to T3874