Skip to content
Snippets Groups Projects
Commit 2b8ce87b authored by Antoine Lambert's avatar Antoine Lambert
Browse files

config: Add LRU cache to get_swh_backend_* functions

I noticed tests in other SWH packages were really slow to execute
since recently and it turned out that the root cause of these
slowdowns is that pytest fixtures like the swh_storage one end up
calling the get_swh_backend_module from swh.core.config.

As that function reads Python package entry points from disk, this
slows down tests execution as it is called before each test using
a fixture as described above.

So add LRU cache to these functions to restore previous execution
times in SWH package tests.
parent bce777a2
No related branches found
Tags v3.6.4
1 merge request!402config: Add LRU cache to get_swh_backend_* functions
Pipeline #12278 passed
# Copyright (C) 2015-2020 The Software Heritage developers
# Copyright (C) 2015-2024 The Software Heritage developers
# See the AUTHORS file at the top-level directory of this distribution
# License: GNU General Public License version 3, or any later version
# See top-level LICENSE file for more information
from copy import deepcopy
from functools import lru_cache
from itertools import chain
import logging
import os
......@@ -332,6 +333,7 @@ def load_from_envvar(default_config: Optional[Dict[str, Any]] = None) -> Dict[st
return cfg
@lru_cache()
def get_swh_backend_module(swh_package: str, cls: str) -> Tuple[str, Optional[type]]:
entry_points = get_entry_points(group=f"swh.{swh_package}.classes")
if not entry_points:
......@@ -358,6 +360,7 @@ def get_swh_backend_module(swh_package: str, cls: str) -> Tuple[str, Optional[ty
return entry_point.module, BackendCls
@lru_cache()
def get_swh_backend_from_fullmodule(
fullmodule: str,
) -> Tuple[Optional[str], Optional[str]]:
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment