This avoids duplication which is growing (right now storage and search proposes
the same pattern in different functions).

Related to T645

Before this commit, decorating with @remote_api_endpoint made
mypy completely unable to detect the type of the decorated function.

This will allow typing paginated endpoints across our modules (swh.storage,
swh.search, swh.indexer, swh.scheduler, ...).

Related to T645

Related to D3460

Related to T2105

Ideally we should refactor those entirely.

Should definitely fix the debian build [1]

[1] https://jenkins.softwareheritage.org/job/debian/job/packages/job/DCORE/job/gbp-buildpackage/118/console

This fixes the debian build yet to come

Instead of the brittle csv we used to use, use postgresql text mode to transfer
data in the `copy` operation. Reference all relevant bits of the PostgreSQL
documentation inline with the code.

This also adds test coverage for most supported types, as well as making sure
all escaping corner cases are covered.

This fixes the ci build [1]

[1] https://jenkins.softwareheritage.org/job/DCORE/job/tests/844/console

There's no reason to allow this, and it's a potential source of
invalid data.

Related to T2367

msgpack only has built-in support for ints that fit in 64 bits. However, we
happen to be storing arbitrary json in the archive, which itself has support for
integers of arbitrary length, which themselves are mapped to "long" integers in
Python, which make the msgpack encoder blow up.

Fortunately, overflowing integers are passed to the default object hook. We
generate a msgpack "extended type" with code 1 for arbitrary integers.

I forgot to run it on files at the root in the previous commit.

The next commit will enable Black; which would break this test,
as lineno() no longer matches the right line.

The contents of the journal up to February 2020 are encoded with this schema, so
we should keep supporting it, until all these objects end up rewritten

msgpack 1.0 disables support for non-str/bytes keys by default, but we use
None as key in the output of swh-storage's snapshot_count_branches;
and we use ints in swh-scheduler.

msgpack now defaults to raw=False.

This commit preserves compatibility with older versions of msgpack.

Some remote api (for instance the scheduler one) does not declare any
extra_encoders in the negotiate decorator. This was raising an error
when querying any declared endpoint.

I forgot to change the async server in
08289ce5.

iso8601 is more strict, and less inefficient.

They will allow swh-storage to override them to serialize and
deserialize swh-model objects.

This will allow swh-storage to pass model objects.

This breaks the current protocol, but deduplicates the code, and will allow
for the addition of extra custom encoders/decoders by callers of this module.

Pros:
* Language-independent
* Works with unpickleable exceptions (eg. cimpl.KafkaException, and
  RLock objects in exceptions raised by tenacity)

Con:
* Shows a less nice error on the client side (if the client is in
  Python), unless the error is whitelisted in `reraise_exceptions`.

 Fixed consistency problems

Optional params are now Optional

Data is now bytes

Rremoving them allows testing the function's signature
matches the existing signature of a specification and type checking.

Moreover, they should not be used by users of the class, so there is no
reason to have them appear in the documentation (generated from
the signature).

This allows initializing Sentry with the environment the process
is running in (ie. "production" or "staging"), which allows filtering
in the web UI to only show issues happening in one of the environments.

https://docs.sentry.io/enriching-error-data/environments/?platform=python

Related to T2188

This enables tracking of issues by release/package version in sentry
(https://docs.sentry.io/workflow/releases/).

Also deduplicates code used to call sentry_sdk.init by making all our
init code go through the same function swh.core.sentry.init_sentry."