David Douard authored 2 years ago and Antoine R. Dumont committed 2 years ago

As a consequence, this introduces the use of swh.journal in the provenance swh
requirements.

This also:
- deprecates the `swh provenance iter-origins`
- in favor of the new `swh provenance origin from-csv` cli
- Add tests around those CLIs ^

Related to T4315

It queries all the underlying archives and returns
the first result found.
It a naive implementation but will allow to have
a fallback the the swh-storage if some is requested
in an outdated swh-graph

Related to T4315

Workaround to support the empty response of swh-graph when a
revision has no parent

In this case, swh-graph should reply a 201 no-content
and this response should be correctly interpreted by the swh-graph
lib, but it the meantime, it'll allow provenance to work with
swh-graph

Related to T4314

For now, it starts the provenance storage + the origin layer part
It's full of shortcuts but it will help to reproduce the problem declared
in the related issue

Related to T3402

Related to T4302

This is tooling around swh-provenance and not provenance itself but, it's:
- not covered
- not declaring its dependencies in the main setup.py
- missing __init__.py from that folder onward
- failing the main build [1]

[1]
https://jenkins.softwareheritage.org/view/swh%20master%20(draft)/job/DPROV/job/tests/706/console

mypy would complain that the result of cursor.fetch*() are tuples and
not mappings.

This fixes current failing master build [1].

[1] https://jenkins.softwareheritage.org/view/swh-draft/job/DPROV/job/tests/623/console

pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for
pytest >= 7 so we can now drop the pytest pinning.

It is not require anymore after simplifying the origin-revision
layer algorithm.

to avoid processing twice the history graph of a revision that was
already seen as the head of another origin.

To install the new hook:

  $ pre-commit install -t commit-msg

Related to T3916

Duplicated entries are now filtered by a `SELECT DISTINCT` clause.

Original `directory_ls` query is cleaned up to avoid retrieving
unnecesary information. Specific query is added for the case where
no file filtering by size is done.