Related to T3922

black is considered stable since release 22.1.0 and the version
we are currently using is quite outdated and not compatible with
click 8.1.0, so it is time to bump it to its latest stable release.

Please note that E501 pycodestyle warning related to line length
is replaced by B950 one from flake8-bugbear as recommended by black.
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Related to T3922

Looks like .tar.xz files just got slightly larger in Sid (Python 3.10 upgrade?),
which makes it go over the limit.

Resolves SWH-DEPOSIT-2P <https://sentry.softwareheritage.org/share/issue/b8509b67972f425a8f5c06805f8cf2fe/>

'metadata_raw' made sense, to discriminate from 'metadata_dict'; but no
longer does, now that the latter was removed.

Additionally, swh-web expects it to be named 'raw_metadata', so it
could never actually get the metadata.

pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for
pytest >= 7 so we can now drop the pytest pinning.

Manually validate <codemeta:affiliation>. Unfortunately, this cannot be
validated by codemeta.xsd, because Codemeta has conflicting requirements:
1. https://codemeta.github.io/terms/ requires it to be Text (represented
   by simple content), but
2. https://doi.org/10.5063/SCHEMA/CODEMETA-2.0 requires it to be an
   Organization (represented by complex content)

See https://github.com/codemeta/codemeta/pull/239 for a discussion about
this issue.

Due to test modules being copied in subdirectories of the
build directory by setuptools, it makes pytest fail by raising
ImportPathMismatchError exceptions when invoked from root
directory of the module.

So ignore the build folder to discover tests.

They are allowed by ISO8601, via schema.org, via codemeta; but not by XML Schema's
date format.

Related to T4013#80910

Broken by swh-core 0.12.0 (777ea186fa8d31a972ccc15abb7c42297e0a399f)

It makes it easier to identify tests when failing + helps git to
make readable diffs.

Prior to this, this was fetching all deposits and then for each deposit, query further
information. Then return results and let the pagination happen.

This now keeps the queryset lazy, the pagination happens and when a page is requested,
this fetches further information on the subset required.

Related to T4020

This behavior was (accidentally) removed in 74d5567b.

And Make it non-empty in 'GET Cont-IRI'.

And remove the legacy <atom:deposit_date> from the 'GET Cont-IRI'.
As it was always empty, there is no need to keep it for backward
compatibility.

This function is only used by server-side API checks. Having it defined
in the main utils module makes the deposit client transitively depend on
Django (via swh.deposit.errors), which does not seem necessary.

For now, this only checks they aren't just a string

This only checks the name is a string.

For now this increases code complexity, but this will allow addition
of other check more easily.

The leading newline prevented textwrap.dedent from removing them.

It was never actually tested...

This fixes crashes when running 'pytest -k migration', because swh/deposit/tests_migration/
lacks a conftest to initialize the database.

This will refuse the metadata-only deposit if the metadata provenance does not match.
This is doing a similar check already done when doing deposit with origin url
mismatching that same (client) provider url.

Related to T3677

We don't have much use for it anymore, let's use ElementTree everywhere
for consistency.

This should ease deposit listing in whatever forms (backend db read or client consuming
deposit listing).

Deposit types stand for:
- meta: metadata-only deposit
- code: content deposit

This commit includes a migration schema script which adds a new column 'type'. The
script is also in charge of migration existing data with the right type values..

Related to T3677

Related to T3677

xmltodict was already on the way out for the deposit, and the latest
libexpat security update broke it entirely when dealing with namespaces,
which means we cannot use it until this is addressed.

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1006317

Functional changes of this commit:

1. No more writes to the 'metadata' jsonb column in the DB (as it
   strongly depends on xmltodict)
2. ServiceDocumentDepositClient always outputs a list of collections,
   instead of None/dict/List[dict] depending on the number of
   collections (artefact of using xmltodict, which is replaced by proper
   parsing)

No one uses that, and it's redundant, as we provide the original XML