Skip to content

utils/init_svn_repo_from_dump: Improve svnadmin load performance

Antoine Lambert requested to merge anlambert/swh-loader-svn:improve-svnadmin-load-performance into master

"svnadmin load" has a --no-flush-to-disk option enabling faster load while being unsafe on power off. This drawback is not an issue for the subversion loader so use that option to significantly improve the performance for loading a repository from a dump file into a directory on the local filesystem.

Some examples of speedup obtained, the more revisions to load the better the speedup is.

$ svnrdump dump --username anonymous https://svn.cryptool.org/CrypTool1 > crypto.dump
* Dumped revision 0.
...
* Dumped revision 3985.

$ svnadmin create crypto_repo

$ time cat crypto.dump | svnadmin load -q crypto_repo/

real    3m52,387s
user    1m42,395s
sys     0m41,272s

$ svnadmin create crypto_repo2

$ time cat crypto.dump | svnadmin load -q --no-flush-to-disk crypto_repo2/

real    2m13,602s
user    1m36,428s
sys     0m34,756s
$ rsvndump --deltas --keep-revnums https://source.sakaiproject.org/svn/jsf/tags/jsf-2.9.1/ > jsf.dump
* Padded revision 1.
...
* Padded revision 119651.
* Dumped revision 119652.

$ svnadmin create jsf_repo

$ time cat jsf.dump | svnadmin load -q jsf_repo/

real    32m31,558s
user    1m4,005s
sys     3m34,383s

$ svnadmin create jsf_repo2

$ time cat jsf.dump | svnadmin load -q --no-flush-to-disk jsf_repo2/

real    1m30,764s
user    0m22,160s
sys     1m6,862s

Merge request reports