Add support for (topological) branches and merges in generate_repo.py

include an example input file (currently not used in actual tests).

Depends on !47 (closed).

---

Migrated from D5892 (view on Phabricator)

Build has FAILED

Patch application report for D5892 (id=21102)

Could not rebase; Attempt merge onto c9d1369b...

Updating c9d1369..413c2a4
Fast-forward
 mypy.ini                                           |   3 +
 requirements-test.txt                              |   1 +
 swh/provenance/tests/conftest.py                   |  38 ++++-----
 swh/provenance/tests/data/cmdbts2.msgpack          | Bin 17734 -> 14470 bytes
 swh/provenance/tests/data/generate_repo.py         |  16 +++-
 .../tests/data/generate_storage_from_git.py        |  86 +++------------------
 swh/provenance/tests/data/out-of-order.msgpack     | Bin 6653 -> 6325 bytes
 swh/provenance/tests/data/repo_with_merges.yaml    |  73 +++++++++++++++++
 8 files changed, 117 insertions(+), 100 deletions(-)
 create mode 100644 swh/provenance/tests/data/repo_with_merges.yaml

Changes applied before test

commit 413c2a48eff28dc2236cbb96b1cc701c62b99897
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 14:52:54 2021 +0200

    Add support for (topological) branches and merges in generate_repo.py
    
    include an example input file (currently not used in actual tests).

commit e1a003a28f2ce1319f65af1bc54dac6003cc4910
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 19:53:06 2021 +0200

    Refactor the generate_storage_from_git dataset creation tool
    
    use the newly introduced swh.journal StreamJournalWriter to generate the
    msgpack-file with the content of the storage serialized, instead of
    reimplementing a similar logic by hand.
    
    Doing so, the msgpack file format is a little bit different, so current
    datasets need to be regenerated.
    
    From the 'consumer' side (the fill_storage fixture), simply use
    swh.storage's process_replay_objects (once again, instead of reinventing
    the wheel).
    
    This is needed because we will need to handle much more model object types
    for testing the origin-revision layer.

Link to build: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/156/ See console output for more information: https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/156/console

Build is green

Patch application report for D5892 (id=21102)

Could not rebase; Attempt merge onto 8ff1ab58...

Merge made by the 'recursive' strategy.
 mypy.ini                                           |   3 +
 requirements-test.txt                              |   1 +
 swh/provenance/tests/conftest.py                   |  38 ++++-----
 swh/provenance/tests/data/cmdbts2.msgpack          | Bin 17734 -> 14470 bytes
 swh/provenance/tests/data/generate_repo.py         |  16 +++-
 .../tests/data/generate_storage_from_git.py        |  86 +++------------------
 swh/provenance/tests/data/out-of-order.msgpack     | Bin 6653 -> 6325 bytes
 swh/provenance/tests/data/repo_with_merges.yaml    |  73 +++++++++++++++++
 8 files changed, 117 insertions(+), 100 deletions(-)
 create mode 100644 swh/provenance/tests/data/repo_with_merges.yaml

Changes applied before test

commit 4f26e661d3f66dd68025184875c8b628fd650402
Merge: 8ff1ab5 413c2a4
Author: Jenkins user <jenkins@localhost>
Date:   Fri Jun 18 10:32:24 2021 +0000

    Merge branch 'diff-target' into HEAD

commit 413c2a48eff28dc2236cbb96b1cc701c62b99897
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 14:52:54 2021 +0200

    Add support for (topological) branches and merges in generate_repo.py
    
    include an example input file (currently not used in actual tests).

commit e1a003a28f2ce1319f65af1bc54dac6003cc4910
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 19:53:06 2021 +0200

    Refactor the generate_storage_from_git dataset creation tool
    
    use the newly introduced swh.journal StreamJournalWriter to generate the
    msgpack-file with the content of the storage serialized, instead of
    reimplementing a similar logic by hand.
    
    Doing so, the msgpack file format is a little bit different, so current
    datasets need to be regenerated.
    
    From the 'consumer' side (the fill_storage fixture), simply use
    swh.storage's process_replay_objects (once again, instead of reinventing
    the wheel).
    
    This is needed because we will need to handle much more model object types
    for testing the origin-revision layer.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/159/ for more details.

mentioned in merge request !49 (closed)

mentioned in merge request !45 (closed)

Merge request was accepted

approved this merge request

rebase

Merge request was merged

closed

Build is green

Patch application report for D5892 (id=21161)

Could not rebase; Attempt merge onto 8ff1ab58...

Updating 8ff1ab5..6734fd3
Fast-forward
 mypy.ini                                           |   3 +
 requirements-test.txt                              |   1 +
 swh/provenance/tests/conftest.py                   |  38 ++++-----
 swh/provenance/tests/data/cmdbts2.msgpack          | Bin 17734 -> 14470 bytes
 swh/provenance/tests/data/generate_repo.py         |  16 +++-
 .../tests/data/generate_storage_from_git.py        |  86 +++------------------
 swh/provenance/tests/data/out-of-order.msgpack     | Bin 6653 -> 6325 bytes
 swh/provenance/tests/data/repo_with_merges.yaml    |  73 +++++++++++++++++
 8 files changed, 117 insertions(+), 100 deletions(-)
 create mode 100644 swh/provenance/tests/data/repo_with_merges.yaml

Changes applied before test

commit 6734fd36b87233aa7520c90e7b6ad999d75103cf
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 14:52:54 2021 +0200

    Add support for (topological) branches and merges in generate_repo.py
    
    include an example input file (currently not used in actual tests).

commit 7886bf494ab8dd0405ad391a0ca57eed1ee7149c
Author: David Douard <david.douard@sdfa3.org>
Date:   Thu Jun 17 19:53:06 2021 +0200

    Refactor the generate_storage_from_git dataset creation tool
    
    use the newly introduced swh.journal StreamJournalWriter to generate the
    msgpack-file with the content of the storage serialized, instead of
    reimplementing a similar logic by hand.
    
    Doing so, the msgpack file format is a little bit different, so current
    datasets need to be regenerated.
    
    From the 'consumer' side (the fill_storage fixture), simply use
    swh.storage's process_replay_objects (once again, instead of reinventing
    the wheel).
    
    This is needed because we will need to handle much more model object types
    for testing the origin-revision layer.

See https://jenkins.softwareheritage.org/job/DPROV/job/tests-on-diff/174/ for more details.