setup: Do no expose the pytest-plugin any longer

Defining the pytest-plugin though the pytest-plugin [1] makes it loaded by default. This creates loading issues on modules depending on storage but not on the pytest plugin storage exposes. It was explained in the doc and I did not realize [2]

Instead we'll explicitely define to modules depending on the pytest plugins in their root conftest [3]:

pytest_plugins = [ "swh.storage.pytest_plugin" ]

The following fix remains with the current other modules [4] by:

• adding their deps as swh.storage[testing] if not already.

• drop the eventual duplication introduced in tox.ini for the now apparent wrong reasons ;)

• remove the workaround to avoid loading stuff from storage where not needed (swh-web for one)

• [1] https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points

• [2] https://docs.pytest.org/en/stable/writing_plugins.html#plugin-discovery-order-at-tool-startup

• [3] https://docs.pytest.org/en/stable/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file

• [4] as an example of adaptations D3479 (mercurial loader)

Related to T2484

Test Plan

tox

---

Migrated from D3475 (view on Phabricator)

The documentation should read pytest_plugins = ["swh.storage.pytest_plugin"]. You also need to add that line to your own conftest.py if you're depending on that fixture.

Build has FAILED

Patch application report for D3475 (id=12290)

Rebasing onto c21d0e38...

Current branch diff-target is up to date.

Changes applied before test

commit ae0595f6f3ad5ba80da4a3ba5131ae86f0824c86
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 9 10:11:01 2020 +0200

    setup: Do no expose the pytest-plugin any longer
    
    This creates loading issues on modules depending on storage but not on the
    pytest plugin. Somehow the loading is eager and not lazy.
    
    Instead we'll explicitely define to modules depending on the pytest plugins in
    their root conftest:
    
    ```
    pytest_plugin = [ "swh.storage.pytest_plugin" ]
    ```
    
    - [1]
    https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points
    
    Related to [T2484](https://forge.softwareheritage.org/T2484 'view original for T2484 on Phabricator')

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/425/ See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/425/console

You also need to add that line to your own conftest.py if you're depending on that fixture.

I did not since we import * from the conftest. But yeah, i guess that'd make sense we eat our own cake ;)

I'll adapt my typos in description and commit message. Thanks ;)

Adapt according to remarks:

• Update docstring
• use the pytest_fixtures definition ourselves

! In !945 (closed), @ardumont wrote: Adapt according to remarks:

• Update docstring

what docstring?

what docstring?

oops, sorry "git commit message"

Build has FAILED

Patch application report for D3475 (id=12292)

Rebasing onto c21d0e38...

Current branch diff-target is up to date.

Changes applied before test

commit d94b05009b2f9887d5f005aa4e4fb13b9034ffb6
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 9 10:11:01 2020 +0200

    setup: Do no expose the pytest-plugin any longer
    
    This creates loading issues on modules depending on storage but not on
    the pytest plugin storage exposes. Somehow the loading is eager and not lazy.
    
    Instead we'll explicitely define to modules depending on the pytest plugins in
    their root conftest:
    
    ```
    pytest_plugins = [ "swh.storage.pytest_plugin" ]
    ```
    
    - [1]
    https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points
    
    Related to [T2484](https://forge.softwareheritage.org/T2484 'view original for T2484 on Phabricator')

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/427/ See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/427/console

Rebase on latest master

mentioned in merge request swh-loader-mercurial!117 (closed)

mentioned in merge request swh-scheduler!316 (closed)

Looks good to me.

Merge request was accepted

approved this merge request

Build has FAILED

Patch application report for D3475 (id=12301)

Rebasing onto c3803ef8...

Current branch diff-target is up to date.

Changes applied before test

commit 529259ffce0b4c0173025631ec11c53bd4ad8919
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 9 10:11:01 2020 +0200

    setup: Do no expose the pytest-plugin any longer
    
    This creates loading issues on modules depending on storage but not on
    the pytest plugin storage exposes. Somehow the loading is eager and not lazy.
    
    Instead we'll explicitely define to modules depending on the pytest plugins in
    their root conftest:
    
    ```
    pytest_plugins = [ "swh.storage.pytest_plugin" ]
    ```
    
    - [1]
    https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points
    
    Related to [T2484](https://forge.softwareheritage.org/T2484 'view original for T2484 on Phabricator')

Link to build: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/434/ See console output for more information: https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/434/console

Rebase on latest master

Build is green

Patch application report for D3475 (id=12331)

Rebasing onto de38cd11...

Current branch diff-target is up to date.

Changes applied before test

commit f1bb519de763b17cbf6bb07a2c3ba608fce0c759
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 9 10:11:01 2020 +0200

    setup: Do no expose the pytest-plugin any longer
    
    This creates loading issues on modules depending on storage but not on
    the pytest plugin storage exposes. Somehow the loading is eager and not lazy.
    
    Instead we'll explicitely define to modules depending on the pytest plugins in
    their root conftest:
    
    ```
    pytest_plugins = [ "swh.storage.pytest_plugin" ]
    ```
    
    - [1]
    https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points
    
    Related to [T2484](https://forge.softwareheritage.org/T2484 'view original for T2484 on Phabricator')

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/439/ for more details.

• Move the pytest_plugins variable definition to the root conftest.py
• Rework commit message to align with the diff description

Build is green

Patch application report for D3475 (id=12333)

Rebasing onto de38cd11...

Current branch diff-target is up to date.

Changes applied before test

commit 9465ad8bf400ea58d5290c2ca8d07eaa3af32670
Author: Antoine R. Dumont (@ardumont) <ardumont@softwareheritage.org>
Date:   Thu Jul 9 10:11:01 2020 +0200

    setup: Do no expose the pytest-plugin any longer
    
    Defining the pytest-plugin though the pytest-plugin [1] makes it loaded by default.
    This creates loading issues on modules depending on storage but not on
    the pytest plugin storage exposes. It was explained in the doc and I did not realize [2]
    
    Instead we'll explicitely define to modules depending on the pytest plugins in
    their root conftest [3]:
    
    ```
    pytest_plugins = [ "swh.storage.pytest_plugin" ]
    ```
    
    - [1] https://docs.pytest.org/en/stable/writing_plugins.html#setuptools-entry-points
    
    - [2] https://docs.pytest.org/en/stable/writing_plugins.html#plugin-discovery-order-at-tool-startup
    
    - [3] https://docs.pytest.org/en/stable/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file
    
    Related to [T2484](https://forge.softwareheritage.org/T2484 'view original for T2484 on Phabricator')

See https://jenkins.softwareheritage.org/job/DSTO/job/tests-on-diff/440/ for more details.

Some references in the commit message have been migrated:

• T2484 is now swh/meta#2484 (closed)

Rebase on latest master