bin/install: install packages one by one and enforce prerequisites
This is trying to make bin/install (again) a click-button developer setup installation tool, simplifying the process for newcomers and making it faster for everyone.
To that end:
-
we ensure that pip is up to date before using it for anything else
-
we install the prerequisites (flake8, mypy, etc.) to avoid having to name them one by one in user documentation
-
we install packages one by one, rather them all together, to avoid that pip failures force users to restart the whole process from scratch. In passing, this makes also things much easier for the new pip resolver. To avoid dependency inconsistencies we install packages in bin/pip-swh-packages, which is already supposed to be topologically sorted
-
we also install packages only once, rather than first without -with-testing and later with it (as currently documented in the developer setup), because the "[testing]" markers are extra requires, which always depend on the base package (hopefully)
When this gets landed, the developer setup can be shortened to just say "run bin/install".
bin/pip-swh-packages: remove [extras] markers before topo-sorting
Migrated from D5154 (view on Phabricator)
Merge request reports
Activity
-
Yes, that sounds more reasonable now.
Thanks.
Wondering if that should go into the swh-environment build which seems to try to do something similar.
Aside that this build frequently fails due to some repository being red often) that migth help in detecting misdeclared dependency.
1 1 #!/usr/bin/env bash 2 2 3 pip install $(./bin/pip-swh-packages --with-testing) "$@" 3 # Usage: bin/install [PIP_INSTALL_OPTION..] 4 5 PREREQUISITES="pre-commit flake8 pifpaf tox wheel mypy" 6 7 pip install --upgrade pip "$@" 8 9 pip install $PREREQUISITES "$@" 10 11 bin/pip-swh-packages --with-testing | \ 12 while read pkg_spec ; do 13 pip install $pkg_spec 14 done -
I'm also wondering whether we should add checks on runtime dependencies (the ones that could not be dealt with pip and which are referenced in the docs [1]).
A simple check and warning if missing.
Not a bad idea, but they are all system dependencies, I have no idea if there is a reasonable way to check for their presence without having to discriminate debian/redhat/etc. (which clearly we do not want to do).
I've took care of pre-commit though, but I think it's needed at bin/update time (which happens before), so it's not really helping either.