config: Deprecate SWHConfig in favor of load_from_envvar function

That new function declares exactly what happens today during our configuration loading (for loader/lister/indexer/... and rpc services in general not for the CLIs which will be dealt with later).

Loads and parses the yaml configuration file out of the SWH_CONFIG_FILENAME environment variable. Allowing eventual dict enrichment with default configuration if provided.

Everything docker/staging/production related is setting the SWH_CONFIG_FILENAME in the environment [1-5] (including the tests now as well).

This currently impacts (which I will deal with soon):

$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:class BaseIndexer(SWHConfig, metaclass=abc.ABCMeta):
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/rehash.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/rehash.py:class RecomputeChecksums(SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-lister/swh/lister/core/lister_base.py:class ListerBase(abc.ABC, config.SWHConfig):
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/core/loader.py:class BaseLoader(config.SWHConfig, metaclass=ABCMeta):
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/loader.py:from swh.core.config import SWHConfig
$SWH_ENVIRONMENT_HOME/swh-loader-core/swh/loader/package/loader.py:        self.config = SWHConfig.parse_config_file()

• [1] prod/staging: https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/?grep=SWH_CONFIG_FILENAME
• [2] docker: https://forge.softwareheritage.org/source/swh-environment/browse/master/?grep=SWH_CONFIG_FILENAME

As sample:

• [3] loader/lister: https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/site-modules/profile/templates/swh/deploy/worker/swh-worker@.service.erb$10
• [4] scheduler: https://forge.softwareheritage.org/source/swh-environment/browse/master/docker/docker-compose.yml$109
• [5] api/rpc servers: https://forge.softwareheritage.org/source/puppet-swh-site/browse/production/site-modules/profile/manifests/swh/deploy/rpc_server.pp$104

Related to #1532 (closed) Related to #1410

Test Plan

tox

---

Migrated from D3965 (view on Phabricator)

swh/core/tests/test_config.py

@@ -326,3 +326,39 @@ def test_merge_config_type_error():
             config.merge_configs({"a": v}, {"a": {}})
         with pytest.raises(TypeError):
             config.merge_configs({"a": {}}, {"a": v})
+
+
+def test_load_from_envvar_no_environment_var_swh_config_filename_set():
+    """Without SWH_CONFIG_FILENAME set, load_from_envvar raises"""
+
+    with pytest.raises(AssertionError, match="SWH_CONFIG_FILENAME environment"):
+        config.load_from_envvar()
+
+
+def test_load_from_envvar_no_default_config(swh_config, monkeypatch):
+    config_path = str(swh_config)
+    monkeypatch.setenv("SWH_CONFIG_FILENAME", config_path)
+
+    actual_config = config.load_from_envvar()
+
+    expected_config = config.read(config_path)
+    assert actual_config == expected_config
+
+
+def test_load_from_envvar_with_default_config(swh_config, monkeypatch):
+    default_config = {
+        "number": 666,
+        "something-cool": ["something", "cool"],
+    }
+
+    config_path = str(swh_config)
+    monkeypatch.setenv("SWH_CONFIG_FILENAME", config_path)
+
+    actual_config = config.load_from_envvar(default_config)
+
+    expected_config = config.read(config_path)
+    expected_config.update(
+        {"number": 666, "something-cool": ["something", "cool"],}
+    )
+
+    assert actual_config == expected_config