The test graph is too hard to change

As discussed in !547 or swh-fuse!93 (comment 197559), we need to find a way to test new edges as we find them. The options I see:

1. Add them to the test graph. Downsides: This bloats the Git repository (because we need to recreate 500kB worth of binary files), and requires changing plenty of swh-graph/swh-datasets/swh-provenance tests (ie. it's a breaking change). See !161 (closed) as an example (from before we split these tests across repositories).
2. Add a new test graph every time we find new edge cases. Downsides: This bloats both the Git repository and checkouts (again, 500kB. Or 250kB if we exclude ORC and edge-CSV exports).
3. Generate graphs on the fly, as a test- or session-scoped fixture. This would allow each dependent package to have its own test graphs, and run tests on very specific graph. This increases the test suite runtime (5s of wall/cpu time on my machine for the test graph, so probably about 20s on Jenkins)

added priority:Normal label

mentioned in merge request swh-fuse!93 (merged)

changed the description

I just remembered that I created a way to generate test graphs that are not compressed: https://docs.rs/swh-graph/latest/swh_graph/graph_builder/struct.GraphBuilder.html, which can be dumped to disk and loaded.

This would solve the problem, but I need to document this better, and make it usable from the gRPC server.

actually, it's already supported by the gRPC server, with --graph-format json

mentioned in merge request !726

Solution 1 has two important upsides:

• it's simple
• it would force swh-{graph,datasets,provenance,fuse} to support edge cases any time we add one... and it makes sense. Those application are made for the complete graph, so they must be robust to those cases.

On repository bloat: it already weights more than 200Mb, so 500kb is negligible. But is 200mb worth moving to git annex ?

most of the 200MB is not reachable from the master branch but from debian/* branches, which we can probably clean up now.

it would force swh-{graph,datasets,provenance,fuse} to support edge cases any time we add one... and it makes sense. Those application are made for the complete graph, so they must be robust to those cases.

but it means every single test of swh-provenance must test all the edge cases, even those that are not relevant to swh-provenance. It would make the tests unreadable. And they would all need to be updated every time we make changes to the example dataset; which requires the person making the change to understand (and have in "live memory") what is being tested. I imagine every change to the test graph would take me hours, or a couple of days for someone not familiar with swh-provenance.

most of the 200MB is not reachable from the master branch but from debian/* branches, which we can probably clean up now.

@olasd cleaned up debian/* branches and tags, the swh-graph repo now weighs only 7MB.

Great !

This gives another perspective to !726, now I see what you did there and agree that option 2 is a reasonable choice right now. After all, it's not proven yet that it will increase the repository size to 100Mb (even on the long term !).

What are the advantages of option 2 over #4844 (comment 197578) though?

as far as I understood they're very, very similar, aren't they ?

Ah yes, indeed. The only difference is that the latter does not require checking in somewhat large binary files.

mentioned in issue swh-fuse#2919