In combination with the Pytest 8.1 changes for namespace package
support, this allows us to get rid of the envsitepackagesdir hacks,
which makes the output of all tools much more useful.

Add a pre-push git hook checking with twine tool if swh package can be
successfully uploaded on PyPI when pushing an annotated release tag.

In our tests, we are using some patterns that are not supported by the
`importlib` mode, like importing test modules from one another.

So switch back to the default `prepend` mode, but add the
`consider_namespace_packages` option that was introduced in pytest 8.1
so our `swh` namespace package gets properly detected as such.

For reference, see:
https://docs.pytest.org/en/latest/reference/reference.html#confval-consider_namespace_packages

(commit message from Lunar's fix in swh-alter).

Related to swh/meta#5075.

The idea is to add the location where swh-docs will install the
Makefile.sphinx directly as search path for the makefile when
building the sphinx documentation in tox.

With the modification of docs/Makefile to replace:

  include swh-docs/Makefile.sphinx

by:

  include Makefile.sphinx

this allows to have all the doc building scenarios working relatively
easily, without the need for adding ../..[...] paths here and there.

It should work ok when building:

- the whole doc from the swh-docs repo using the sphinx tox env; in this
  scenario, all the swh packages are installed from pypi and the
  location where to find this Makefile.sphinx if provided to the make
  command in tox (-I).

- the whole doc from the swh-docs repo using the sphinx-dev tox env;
  here, all the swh package source repositories are expected to be in the
  parent directory (swh-environment), and the Makefile.sphinx (included
  in all swh-xxx/docs/Makefile) will be found thanks to the -I argument
  of the make commande in the tox file.

- the doc for a specific swh package using tox; the tox file also set
  the -I option to the correct location where to find this
  Makefile.sphinx file.

- the doc for a specific swh package directly using the make command from
  a venv; here, either the user needs to put the -I option to the make
  command, or simply add a symlink to the swh-docs/Makefile.sphinx in
  swh-xxx/docs; this should be done automatically by
  swh-encvironment/bin/update script.

and document why this and the README files are in the gitignore file.

Jérémy Bobbio (Lunar) authored 1 year ago and David Douard committed 1 year ago

This was used in swh-loader-metadata.

As the swh/__init__.py file was removed in swh packages, isort no longer
detects swh.* packages being in the first party category but in third
party one.

So ensure to restore previous behavior by adding 'swh' in the
known_first_party isort option.

- remove the (mostly) useless sphinx-dev env
- install swh.docs in non-editable mode, and
- use the Makefiles provided by swh.docs as non-package data (in share/)
  instead of depending on the swh.docs sources to be available somewhere
  nearby.

Seems it is actually still needed...

Fix a mixed up have_cli / have_workers in the pyproject jinja template.

Remove `usedevelop=true` options; these are not needed and can generate
a build error on some swh packages.

This is handled in swh-docs directly.

Make black and isort run before flake8 so that black can fix style
issues before flake check for these.

and add a test file to be able to run pytest.

These tox environments will create a test swh project from the template
and run the corresponding test from the generated project.

It's not very pretty, but...

This adds a tox file which simply runs the `copier copy` command; no
other validation is made so far.

See the README file.

It now needs types-click which is indeed a dependency of
swh.core[testing].

GitLab will display the content of the README file when browsing the
repository. But in case the file is a symlink, it will display the
path pointed by the symlink. There is a 6 year old issue about this:
https://gitlab.com/gitlab-org/gitlab/-/issues/15093

We can workaround the issue by having the content at the root of the
repository and a symlink to this file in the `docs/` directory.

Related to swh/meta#4960

Related to swh/meta#4959

`swh-foo` is used through the template as our example module name.
The documentation template used `swh-py-template` instead.
Let’s use `swh-foo` here as well so template users can look for
the string `foo` until they have customized everything.

- Switch from Phrabricator to GitLab.
- Update pre-commit dependencies.
- Update copyright years.

This fixes python 3.7 support due to poetry, a dependency of isort, that
removed support for that Python version in a recent release.

In order to remove warnings about /apidoc/*.rst files being included
multiple times in toc when building full swh documentation, prefer to
include module indices only when building standalone package documentation.

Related to T4496

- pre-commit from 4.1.0 to 4.3.0,
- codespell from 2.2.1 to 2.2.2,
- black from 22.3.0 to 22.10.0 and
- flake8 from 4.0.1 to 5.0.4. Also freeze flake8 dependencies.

Also change flake8's repo config to github (the gitlab mirror
being outdated).

That hook can be frustrating as it can discard a long commit message
if it finds a typo in it so better removing it.

Related to T3922

black is considered stable since release 22.1.0 and the version
we are currently using is quite outdated and not compatible with
click 8.1.0, so it is time to bump it to its latest stable release.

Please note that E501 pycodestyle warning related to line length
is replaced by B950 one from flake8-bugbear as recommended by black.
https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html#line-length

Related to T3922

pytest-postgresql 3.1.3 and pytest-redis 2.4.0 added support for
pytest >= 7 so we can now drop the pytest pinning.

Due to test modules being copied in subdirectories of the
build directory by setuptools, it makes pytest fail by raising
ImportPathMismatchError exceptions when invoked from root
directory of the module.

So ignore the build folder to discover tests.

To install the new hook:

  $ pre-commit install -t commit-msg