When shutil.unpack_archive failed to unpack a tarball, fallback using
the tar command to perform that task.

Such issues were encountered when trying to unpack old tarballs coming
from CPAN.

Related to T2833

It exist cases when an uncompressed tarball has directories without
any traversal or read permissions so ensure to also fix directory
permissions in normalize_permissions function.

Related to T3781

This will allow to reuse it when we try to detect tarball nature in lister.

Related to T3781

This mirrors the behavior of the synchronous server.

In practice, this means the objstorage won't log ObjNotFoundError or
report them to Sentry.

In order for that useful fixture to be shared across multiple swh
packages, move it from swh-indexer to main swh-core pytest plugin.

Also add a couple of tests for it.

Without the return, 'psycopg2.connect(db_or_conninfo)' is actually reachable
when calling next() twice on the generator

This will be used to make the storage retry proxy retry harder
when the storage server raises errors like psycopg2.OperationalError;
which last longer than we want the proxy to retry for typical
exceptions.

It was removed from all RPC servers years ago.

They are client errors, therefore it does not make sense to log them
as exceptions (which is only useful for debugging the server).

According to
<https://github.com/pallets/flask/issues/4734#issuecomment-1204124984>,
app.test_client() should not be used as a long-lived context manager.

This causes swh-indexer's tests to break with Flask >= 2.2.0 because
some of them start threads within the lifespan of the context manager,
which causes a crash.

As far as I can tell, simply assigning app.test_client() instead of
entering does not break anything.

so one can create database (e.g. for unit tests) of given flavor.

We already have 'msgpack >= 1.0.0' in requirements.txt

_zst_unpack method added to support '.tar.zst' extension and 'application/zstd', 'application/x-zstd' mime type.
Add archive example file and a test for unpacking zst file.

It exists many cases where using unzip to extract a jar archive
fails while using the zipfile module succeeds.

So prefer to use the zipfile module to uncompress jar archives.

Related to T4318

Discovered when working on [1].

Related to T4228

[1] D7949

This now automatically grant read-only access to the guest user at db initialization or
upgrade time. This way, sysadm no longer have to manually alter the schema after
initialization or upgrade.

Related to T4228

The sample of those exotic urls got extracted out the staging scheduler [1].

And the actual run of that code makes the github api (without mocks) returns the correct
canonical urls [2]

Related to T3874

[1] P1371

[2]
```
$ ipython
...
In [1]: from swh.core.github.utils import get_canonical_github_origin_url

In [2]: get_canonical_github_origin_url('git@github.com/huaweicloud/huaweicloud-sdk-java-v3.git')
No tokens set in configuration, using anonymous mode
Out[2]: 'https://github.com/huaweicloud/huaweicloud-sdk-java-v3'

In [3]: get_canonical_github_origin_url('git//github.com/powertac/powertac-server.git')
No tokens set in configuration, using anonymous mode
Out[3]: 'https://github.com/powertac/powertac-server'

In [4]: get_canonical_github_origin_url('https://${env.GITHUB_USER}:${env.GITHUB_TOKEN}@github.com/molgenis/vibe.git')
No tokens set in configuration, using anonymous mode
Out[4]: 'https://github.com/molgenis/vibe'

In [5]: get_canonical_github_origin_url('ssh://git@github.com/softwaremagico/ThinkMachine.git')
No tokens set in configuration, using anonymous mode
Out[5]: 'https://github.com/softwaremagico/ThinkMachine'

In [6]: get_canonical_github_origin_url('ssh://github.com:alibaba/SmartEngine.git')
No tokens set in configuration, using anonymous mode
Out[6]: 'https://github.com/alibaba/SmartEngine'

In [7]: get_canonical_github_origin_url('//github.com:networknt/light-tram-kafka.git')
No tokens set in configuration, using anonymous mode
Out[7]: 'https://github.com/networknt/light-tram-kafka'

In [8]: get_canonical_github_origin_url('[fetch=]git@github.com:turnonline/ecosystem-admin-widgets.git')
No tokens set in configuration, using anonymous mode
Out[8]: 'https://github.com/turnonline/ecosystem-admin-widgets'

In [9]: get_canonical_github_origin_url('git@github.com:ttulka/spring-boot-configuration-properties-store.git')
No tokens set in configuration, using anonymous mode
Out[9]: 'https://github.com/ttulka/spring-boot-configuration-properties-store'

```

to prepare the deprecation of db/pytest_plugin.py; it's the only
function from db/pytest_plugin.py still in use.

instead of the custom factory_fact.

This is the second part of 240ae09d.

Related to T4228

Some module cannot just define their module db configuration key entry as their module
name because they define more services than just their db (e.g. indexer, scrubber, ...)

Related to T4284

Related to T4284

Related to T4232

Related to T4232

This also:
- sandbox the new module into its own package to simplify packaging

Related to T4232

With latest version of hypothesis, this test often raises the following
error:

hypothesis.errors.FailedHealthCheck: Data generation is extremely slow ...

Setting the number of hypothesis examples for that test to a lower value
makes the error goes away.

This should also fix that test execution in the swh-environment/tests
Jenkins job which often fails.

This also:
- extracts pytest fixtures needed to cover the GitHubSession code.
- avoid mutating the inputted credentials passed in constructors
- rework test docstrings to "it should" format for more concise description

Related to T4232

This new code is within a new arborescence as some more code will get moved alongside in
a new commit (the new session github currently in swh.lister module).

That current code is making anonymous requests to the github api for now.

Related to T4232

adding a swh/__main__.py file.

This allows to reduce the boilerplate regarding initial connection failures.

types-psycopg2 2.9.12 contains this commit:
https://github.com/python/typeshed/commit/506be4fb0adcf38af77325258af337772abcde77

which removes `Any` as the return type of the last variant, allowing
mypy to type-check the returned value.
However, mypy 0.920 only considers the last variant, even though we use
the first one; causing the return type to be badly inferred, and
erroring every time we use it as context manager.

That hook can be frustrating as it can discard a long commit message
if it finds a typo in it so better removing it.