Skip to content

config: Add LRU cache to get_swh_backend_* functions

I noticed tests in other SWH packages were really slow to execute since recently and it turned out that the root cause of these slowdowns is that pytest fixtures like the swh_storage one end up calling the get_swh_backend_module from swh.core.config.

As that function reads Python package entry points from disk, this slows down tests execution as it is called before each test using a fixture as described above.

So add LRU cache to these functions to restore previous execution times in SWH package tests.

Some example timings of tests below:

  • swh-loader-git:
    • before 181 passed, 50 warnings in 122.88s (0:02:02)
    • after 181 passed, 50 warnings in 55.64s
  • swh-loader-svn:
    • before 323 passed, 1 xfailed, 1 xpassed, 1432 warnings in 221.46s (0:03:41)
    • after 323 passed, 2 xpassed, 1432 warnings in 73.37s (0:01:13)
  • swh-deposit:
    • before 355 passed, 43 warnings in 1050.79s (0:17:30)
    • after 355 passed, 43 warnings in 109.57s (0:01:49)
Edited by Antoine Lambert

Merge request reports

Loading