and add -j4 to the 'mr run pre-commit install' comand.

Related to swh/meta#5075.

This should fix the building of the doc for a single swh package using
a simple `make` command.

- Add copier as a dependency
- Adapt bin/init-py-repo to:
  - use copier to initialize the new swh project
  - create and initialize the git repo
  - optionnally create the corresponding gitlab project
- Remove the bin/cherry-py-repo-from-template since this is now the job
  of copier itself.
- Adapt the main Makefile to add '-I swh-docs' when needed to build the
  doc from sources.

As mypy does not support editable installs for projects that have a
pyproject.toml and use a build backend that supports PEP 660, force
the legacy behavior of editable installs by passing the configuration
settings "editable_mode=compat" to the build backend.

It enables to remove the pip version restriction in the swh venv and
avoid bad surprises when the mypy pre-commit hook is called when
committing in a swh repository.

See https://github.com/python/mypy/issues/13392 for more details.

Use the same mypy version as in tox files for swh packages.

Related to swh/meta#4960

This script does the following:
 - check if the repo is hosted in the SWH GitLab
 - check if the repo is already forked, then forks it
 - if requested, adds Jenkins as Developer
 - if requested, sets up a `forked` remote to the forked repository

Recent versions of pip can now properly install all swh packages
in editable mode plus their dependencies in a single command.

Previously a pip install command was issued for each swh package.

pip 21.3 added support to editable installs for projects that have a
pyproject.toml and use a build backend that supports PEP 660, but mypy
does not detect yet such editable installs and following errors are
reported when type checking swh modules:

  Cannot find implementation or library stub for module named swh...

So temporarily pin pip until mypy properly supports PEP 660 editable
installs.

Without this, the debian/upstream branch was not really dropped. Failing to repeat the
complete cycle.

```
...
+ git checkout v0.1.0
HEAD is now at 04b9a34 Add a non-optimized non-incremental Bazaar loader
+ git clean -dfx
Removing .eggs/
Removing build/
Removing swh.loader.bzr.egg-info/
+ git checkout -b debian/upstream
fatal: A branch named 'debian/upstream' already exists.
```

Related to T3915

 - bin/debpkg-get-dependencies: update for recent versions of dh-python
 - bin/debpkg-update-automatic-dependencies: new script to update
 debian/control from the output of debpkg-get-dependencies
 - bin/debpkg-bootstrap-branches: generate a packaging branch out of a
 single tag, using files from the `debian-template` directory

This is both a temporary fix for https://github.com/pypa/pip/issues/10573 ,
which currently breaks entirely our developer seutp, and also a saner approach
to future pip breakages.

As part of this, refactor a bit pip invocation in bin/install to have flags and
invocation logic in a single place.

This is trying to make bin/install (again) a click-button developer setup
installation tool, simplifying the process for newcomers and making it faster
for everyone.

To that end:

- we ensure that pip is up to date before using it for anything else

- we install the prerequisites (flake8, mypy, etc.) to avoid having to name
them one by one in user documentation

- we install packages one by one, rather them all together, to avoid that pip
failures force users to restart the whole process from scratch. In passing,
this makes also things much easier for the new pip resolver. To avoid
dependency inconsistencies we install packages in bin/pip-swh-packages, which
is already supposed to be topologically sorted

- we also install packages only *once*, rather than first without -with-testing
and later with it (as currently documented in the developer setup), because the
"[testing]" markers are extra requires, which always depend on the base
package (hopefully)

When this gets landed, the developer setup can be shortened to just say "run
bin/install".

If there were no spaces between the dependency name and its pinned version,
the latter was not removed by the getdeps function.

It kind of duplicates mr's features, but it adds some sanity checks
(clean checkout, master branch, showing the diff on each repo)

also ship bin/install as a script to only do the installation

Antoine Eiche authored 5 years ago and Antoine R. Dumont committed 5 years ago

Summary:
NixOS (like some other distributions) doesn't have bash installed in
/bin.
This would also allow users to use the $PATH to choose a
alternative bash version.

Reviewers: #reviewers, ardumont, vlorentz, olasd

Reviewed By: #reviewers, ardumont, vlorentz, olasd

Subscribers: seirl, olasd, zack, douardda, vlorentz, ardumont

Differential Revision: https://forge.softwareheritage.org/D2793

these should now be useless.

Some dependencies are of the form `module[a,b]`. Prior to this commit,
this resulted in dependencies of the form
`module[a,b][testing]`. Those are not understood by pip.

This fixes that behavior to merge the extra testing category
dependency: `module[a,b,testing]`

Related T1454