Skip to content

Fix packaging and debian build

Antoine R. Dumont requested to merge ardumont/swh-storage:fix-package into master

Some separated commits:

  • Break the recurisve fixture definition which is possibly a source of issue in the debian build

  • setuptools: Include only swh packages when building distribution archive (there are warhings about it which keep on distracting me when trying to solve the main issue)

Currently, latest setuptools release makes the find_packages raises when building the distribution archive. It mentions that the folders sql, sql/upgrade, etc... are imported but not explicitly declared as packages [1] [2].

According to the documentation, this builds the archive as we want it (as before including the sql folders).

Reproduced in venv locally without the commit, it fails with the same message as [2] (see [4]) and with it, it's no longer complaining [5].

[1]

Python recognizes 'swh.storage.sql' as an importable package[^1],
10:32:14          but it is absent from setuptools' `packages` configuration.
...

[2] https://jenkins.softwareheritage.org/view/swh-debian%20(draft)/job/debian/job/packages/job/DSTO/job/gbp-buildpackage/468/console

[3] https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#finding-namespace-packages

[4]

$ python setup.py sdist build
/home/tony/work/inria/repo/swh/.direnv/python-3.10/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'swh.storage.sql' as data is deprecated, please list it in `packages`.
    !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'swh.storage.sql' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    'swh.storage.sql' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that 'swh.storage.sql' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)
/home/tony/work/inria/repo/swh/.direnv/python-3.10/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'swh.storage.sql.logical_replication' as data is deprecated, please list it in `packages`.
    !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'swh.storage.sql.logical_replication' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    'swh.storage.sql.logical_replication' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that 'swh.storage.sql.logical_replication' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)
/home/tony/work/inria/repo/swh/.direnv/python-3.10/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'swh.storage.sql.upgrades' as data is deprecated, please list it in `packages`.
    !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'swh.storage.sql.upgrades' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    'swh.storage.sql.upgrades' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that 'swh.storage.sql.upgrades' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)
/home/tony/work/inria/repo/swh/.direnv/python-3.10/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'swh.storage.tests.data' as data is deprecated, please list it in `packages`.
    !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'swh.storage.tests.data' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    'swh.storage.tests.data' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that 'swh.storage.tests.data' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

  check.warn(importable)
/home/tony/work/inria/repo/swh/.direnv/python-3.10/lib/python3.10/site-packages/setuptools/command/build_py.py:202: SetuptoolsDeprecationWarning:     Installing 'swh.storage.tests.migrate_extrinsic_metadata' as data is deprecated, please list it in `packages`.
    !!


    ############################
    # Package would be ignored #
    ############################
    Python recognizes 'swh.storage.tests.migrate_extrinsic_metadata' as an importable package,
    but it is not listed in the `packages` configuration of setuptools.

    'swh.storage.tests.migrate_extrinsic_metadata' has been automatically added to the distribution only
    because it may contain data files, but this behavior is likely to change
    in future versions of setuptools (and therefore is considered deprecated).

    Please make sure that 'swh.storage.tests.migrate_extrinsic_metadata' is included as a package by using
    the `packages` configuration field or the proper discovery methods
    (for example by using `find_namespace_packages(...)`/`find_namespace:`
    instead of `find_packages(...)`/`find:`).

    You can read more about "package discovery" and "data files" on setuptools
    documentation page.


!!

Refs. swh/infra/sysadm-environment#4997 (closed)

Edited by Antoine R. Dumont

Merge request reports