- Nov 12, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
- Nov 11, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
- Nov 09, 2019
-
-
Stefano Zacchiroli authored
... and it is brittle enough to make the overall doc build fail fairly often. We will reconsider building it when we can actually shipped it (see T1971)
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
fix docs.s.o build failure
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
- Nov 08, 2019
-
-
Stefano Zacchiroli authored
This avoids that the MPH step loads the full (decompressed!) nodes file in memory. To achieve this, force usage of /bin/bash as shell to run the various steps.
-
- Nov 07, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
before this change it wasn't interpolated in case java_tool_options was given in configuration (and contained '{logback}')
-
Stefano Zacchiroli authored
- add logging for begin/end of loading steps (.mph, .order) - add logging of local speed for pid->node, because average speed might be skewed by temporary sort hangs
-
- Nov 06, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
no longer needed extra indirection
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
purely cosemtic, no functional change
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-
- Nov 05, 2019
-
-
Stefano Zacchiroli authored
This floors the maximum amount of RAM that will be used for this step, avoiding OOM kills. By relying on GNU sort we use an industry-grade tool for this kind of stuff, paging to disk as needed. Closes T1950
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
this allows to be more flexible in how steps are implemented, which is gonna come in handy when we change compression format for nodes/edges files
-
Stefano Zacchiroli authored
For naming uniformity with the Java side, that uses "node" for integer node IDs everywhere. Before this change it was really confusing to have commands like "swh map dump -t int2pid" to generate files like "foo.node2pid.bin".
-
- Nov 04, 2019
-
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
this subsumes the previous tools/migrations/ used to migrate from CSV to binary maps
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
Before this change we had an off-by-1: Java type integer IDs were 0-based, Python ones 1-based. With this change they match and are both 0-based. WARNING: with this change we break backward compatibility for the Python client when reading binary maps that were generated (via a Python hack) before this change. They will need to be regenerated either using the now available Java-based generation of binary maps or by rerunning the Python hack with the new code.
-
Stefano Zacchiroli authored
-
Stefano Zacchiroli authored
-