swh.indexer.storage: Unify get_indexer_storage function with others

Trying to sense a bit better on how to converge with what's described in the hackmd discussion [1]

(also, we have those kinds of tests on at least the storage and search).

Impacts [2]:

• D4272: swh-indexer

• swh-web!1020 (closed): swh-web

• [1] https://hackmd.io/8hxTL4XMQoO2RVKtqFqM2g?view#Synthesis-of-discussion-tenmadouarddaolasd-2020-10-12

• [2]

$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/cli.py:    from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/cli.py:        get_indexer_storage, ctx.obj["config"], "indexer_storage", indexer_storage_url
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:from swh.indexer.storage import INDEXER_CFG_KEY, PagedResult, Sha1, get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/indexer.py:        self.idx_storage = get_indexer_storage(**idx_storage)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/conftest.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/conftest.py:    idx_storage = get_indexer_storage("memory")
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_api_client.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_api_client.py:    server.storage = get_indexer_storage("local", db=swh_indexer_storage_postgresql.dsn)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/conftest.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/conftest.py:    return get_indexer_storage("local", db=swh_indexer_storage_postgresql.dsn)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:from swh.indexer.storage import IndexerStorage, get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:def test_init_get_indexer_storage_failure():
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:        get_indexer_storage("unknown-idx-storage")
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:def test_init_get_indexer_storage(class_name, expected_class, kwargs, mock_psycopg2):
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:        concrete_idx_storage = get_indexer_storage(class_name, **kwargs)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:        concrete_idx_storage = get_indexer_storage(class_name)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:def test_init_get_indexer_storage_deprecation_warning(
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_init.py:        concrete_idx_storage = get_indexer_storage(class_name, args=kwargs)
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_in_memory.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/tests/storage/test_in_memory.py:    return get_indexer_storage("memory")
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/storage/api/server.py:from swh.indexer.storage import INDEXER_CFG_KEY, get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/storage/api/server.py:        storage = get_indexer_storage(**app.config[INDEXER_CFG_KEY])
$SWH_ENVIRONMENT_HOME/swh-indexer/swh/indexer/storage/__init__.py:def get_indexer_storage(cls: str, **kwargs) -> IndexerStorageInterface:
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/config.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/config.py:        swhweb_config["indexer_storage"] = get_indexer_storage(
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/data.py:from swh.indexer.storage import get_indexer_storage
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/tests/data.py:    idx_storage = get_indexer_storage("memory")

Related to swh-core#1410

Test Plan

tox

---

Migrated from D4272 (view on Phabricator)

Build has FAILED

Patch application report for D4272 (id=15109)

Rebasing onto c6afd9d7...

Current branch diff-target is up to date.

Changes applied before test

commit 255c1961d615049ee2b19e819793a7b9a20d3230
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 15:42:58 2020 +0200

    swh.indexer.storage: Unify get_indexer_storage function with others
    
    Related to swh/devel/swh-core#1410

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/88/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/88/console

Fix one instantiation (remains the test-types to fix)

Build has FAILED

Patch application report for D4272 (id=15111)

Rebasing onto c6afd9d7...

Current branch diff-target is up to date.

Changes applied before test

commit fe522f0782e5c18f74e2b6add5b29ef8d0e5ff95
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 15:42:58 2020 +0200

    swh.indexer.storage: Unify get_indexer_storage function with others
    
    Related to swh/devel/swh-core#1410

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/90/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/90/console

mentioned in issue swh-core#1410

Merge request was accepted

approved this merge request

• Add some more assertions
• Simplify instantiation with the new code

TODO:

• Remains the test_type to fix (no clue so far)

Build has FAILED

Patch application report for D4272 (id=15115)

Rebasing onto c6afd9d7...

Current branch diff-target is up to date.

Changes applied before test

commit db4af8e674ece6b00df0cb747a6accd8d26aa550
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 15:42:58 2020 +0200

    swh.indexer.storage: Unify get_indexer_storage function with others
    
    Related to swh/devel/swh-core#1410

Link to build: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/91/ See console output for more information: https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/91/console

• Move test_types into test_init test module
• Fix that test

Some references in the commit message have been migrated:

• T1410 is now swh-core#1410

Build is green

Patch application report for D4272 (id=15117)

Rebasing onto c6afd9d7...

Current branch diff-target is up to date.

Changes applied before test

commit 07c96743d726b668d96d847f1f3297680c780bb7
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Oct 15 15:42:58 2020 +0200

    swh.indexer.storage: Unify get_indexer_storage function with others
    
    Related to swh/devel/swh-core#1410

See https://jenkins.softwareheritage.org/job/DCIDX/job/tests-on-diff/92/ for more details.

Merge request was merged

closed