Skip to content
Snippets Groups Projects

scheduler: Type and unify get_scheduler factory with other factories

Closed scheduler: Type and unify get_scheduler factory with other factories
generated-differential-D4284-source into generated-differential-D4284-target
1 unresolved thread
Closed Antoine R. Dumont requested to merge generated-differential-D4284-source into generated-differential-D4284-target
1 unresolved thread

Impacts:

$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/config.py:        self.scheduler: SchedulerInterface = get_scheduler(**self.config["scheduler"])
$SWH_ENVIRONMENT_HOME/swh-deposit/swh/deposit/tests/conftest.py:        scheduler = get_scheduler(**cfg["scheduler"])
$SWH_ENVIRONMENT_HOME/swh-lister/swh/lister/core/lister_base.py:        self.scheduler = get_scheduler(**self.config["scheduler"])
$SWH_ENVIRONMENT_HOME/swh-scheduler/swh/scheduler/api/server.py:        scheduler = get_scheduler(**app.config["scheduler"])
$SWH_ENVIRONMENT_HOME/swh-scheduler/swh/scheduler/__init__.py:def get_scheduler(cls: str, **kwargs) -> SchedulerInterface:
... # snip scheduler impacts dealt with here
$SWH_ENVIRONMENT_HOME/swh-vault/swh/vault/__init__.py:        args["scheduler"] = get_scheduler(**args["scheduler"])
$SWH_ENVIRONMENT_HOME/swh-web/swh/web/config.py:        swhweb_config["scheduler"] = get_scheduler(**swhweb_config["scheduler"])

Note: Those should still work ok, only a deprecation warning should show up

Related to swh-core#1410

Depends on !157 (closed)

Migrated from D4284 (view on Phabricator)

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4284 (id=15145)

    Rebasing onto 315a2c9d...

    Current branch diff-target is up to date.
    Changes applied before test
    commit c64796332ca7bc9e99745589a49a3adb610eb379
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:12:27 2020 +0200

    scheduler: Type and unify get_scheduler factory with other factories
    
    Related to swh/devel/swh-core#1410

    See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/64/ for more details.

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer
    • Rename to BACKEND_TYPES
    • Adapt docstring according to review
    • Fix one config left in the old format
  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4284 (id=15149)

    Could not rebase; Attempt merge onto 315a2c9d...

    Updating 315a2c9..4ac463b
Fast-forward
 requirements.txt                          |  1 +
 swh/scheduler/__init__.py                 | 57 +++++++++++++++-------
 swh/scheduler/interface.py                |  5 +-
 swh/scheduler/pytest_plugin.py            |  2 +-
 swh/scheduler/tests/es/conftest.py        |  2 +-
 swh/scheduler/tests/test_cli_task_type.py |  2 +-
 swh/scheduler/tests/test_init.py          | 78 +++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_server.py        | 34 +++++---------
 8 files changed, 137 insertions(+), 44 deletions(-)
 create mode 100644 swh/scheduler/tests/test_init.py
    Changes applied before test
    commit 4ac463b11f06a2f0ae924d66b82f5f7c36edf8fc
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:12:27 2020 +0200

    scheduler: Type and unify get_scheduler factory with other factories
    
    Related to swh/devel/swh-core#1410

commit 4fcd407139502035759bee9268cc5499375de356
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:35:54 2020 +0200

    test_server: Simplify exception manipulations

    See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/66/ for more details.

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    Rebase

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4284 (id=15151)

    Could not rebase; Attempt merge onto 315a2c9d...

    Updating 315a2c9..5e2f8dc
Fast-forward
 requirements.txt                          |  1 +
 swh/scheduler/__init__.py                 | 57 +++++++++++++++-------
 swh/scheduler/interface.py                |  5 +-
 swh/scheduler/pytest_plugin.py            |  2 +-
 swh/scheduler/tests/es/conftest.py        |  2 +-
 swh/scheduler/tests/test_cli_task_type.py |  2 +-
 swh/scheduler/tests/test_init.py          | 78 +++++++++++++++++++++++++++++++
 swh/scheduler/tests/test_server.py        | 42 +++++++----------
 8 files changed, 143 insertions(+), 46 deletions(-)
 create mode 100644 swh/scheduler/tests/test_init.py
    Changes applied before test
    commit 5e2f8dc07ccc025e282a8629eb1107d09e32b226
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:12:27 2020 +0200

    scheduler: Type and unify get_scheduler factory with other factories
    
    Related to swh/devel/swh-core#1410

commit dd33cdcfd266f80cb94eb9b3a32fc2c3f2f63592
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:35:54 2020 +0200

    test_server: Simplify exception manipulations

    See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/68/ for more details.

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    Align rpc server configuration as well

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4284 (id=15153)

    Rebasing onto dd33cdcf...

    Current branch diff-target is up to date.
    Changes applied before test
    commit 4657cf5b6cfc282615a149decfbed940f71eb96e
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:12:27 2020 +0200

    scheduler: Type and unify get_scheduler factory with other factories
    
    Related to swh/devel/swh-core#1410

    See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/69/ for more details.

  • Tenma
    Tenma @tenma started a thread on the diff
swh/scheduler/tests/test_init.py 0 → 100644
9
10 from swh.scheduler import get_scheduler
11 from swh.scheduler.api.client import RemoteScheduler
12 from swh.scheduler.backend import SchedulerBackend
13 from swh.scheduler.interface import SchedulerInterface
14
15 SERVER_IMPLEMENTATIONS = [
16 ("remote", RemoteScheduler, {"url": "localhost"}),
17 ("local", SchedulerBackend, {"db": "something"}),
18 ]
19
20
21 @pytest.fixture
22 def mock_psycopg2(mocker):
23 mocker.patch("swh.scheduler.backend.psycopg2.pool")
24
  • Tenma
    Tenma @tenma ·

    Event if useful, this is very surprising. One hardly expects a mock fixture to return a mocker instead of the mock!

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    well, i "injected" the mock_psycopg2 by passing it as parameter so i guess my reasoning was "that's probably needed something not None". I did not check if having None returned is a viable solution.

    It's used indirectly without specifically needing to manipulate that variable. I could have also used:

    @pytest.usefixture("mock_psycopg2")
def test_init_get_scheduler(class_name, expected_class, kwargs):

    I did not check if i could just return None.

  • Please register or sign in to reply
  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    Drop unneeded instruction

  • Software Heritage Continuous Integration for public repositories
    Software Heritage Continuous Integration for public repositories @swh-public-ci ·

    Build is green

    Patch application report for D4284 (id=15174)

    Rebasing onto dd33cdcf...

    Current branch diff-target is up to date.
    Changes applied before test
    commit 13dcaddbed8125a2b722a8ca1186853fd77e9cba
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Fri Oct 16 13:12:27 2020 +0200

    scheduler: Type and unify get_scheduler factory with other factories
    
    Related to swh/devel/swh-core#1410

    See https://jenkins.softwareheritage.org/job/DSCH/job/tests-on-diff/70/ for more details.

  • Tenma mentioned in merge request swh-environment!219 (closed)

    mentioned in merge request swh-environment!219 (closed)

  • Tenma
    Tenma @tenma ·

    Merge request was accepted

  • Tenma approved this merge request

    approved this merge request

  • Antoine R. Dumont
    Antoine R. Dumont @ardumont ·
    Author Maintainer

    Merge request was merged

  • Antoine R. Dumont closed

    closed

    • Please register or sign in to reply