Skip to content

luigi/compressed_graph: Tune -Xmx per task

vlorentz requested to merge vlorentz/swh-graph:tune-mem into master

Instead of passing '-Xmx 3000G' to every task, which prevented running tasks in parallel or to use Luigi's ability to schedule other memory-hungry tasks at the same time while respecting memory limits

This was tested on both the trivial test dataset and on a dataset with only origin and snapshot objects

Resolves #4767 (closed)

Part of #4775

Merge request reports