tarball: Add support for tar.Z, tar.x, tar.lz files

This will allow to deal with more tarballs (gnu, cran, nix, guix, etc...).

Implementation wise, this uses the shutil high-level standard python library.

Feel free to propose a better way of doing this, I did not see any.

Note: I expected the ci build to fail. One test needs the "optional" lzip dependency installed. It did not but... I don't know how to idiomatically make this setup optional (depending on the runtime's current setup). That is without hard-coding check on /usr/bin/lzip for example, etc... And i'd rather we do that so any hint is welcome ;)

Related to swh-loader-core#2110 (closed)

Test Plan

tox

---

Migrated from D2371 (view on Phabricator)

Build is green See https://jenkins.softwareheritage.org/job/DCORE/job/tox/363/ for more details.

Use a smaller .tar.lz sample file

Build is green See https://jenkins.softwareheritage.org/job/DCORE/job/tox/364/ for more details.

• core.tarballs: Improve docstring

Build is green See https://jenkins.softwareheritage.org/job/DCORE/job/tox/365/ for more details.

Merge request was returned for changes

Adapt according to review

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tox/366/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tox/366/console

Fix missing assertion adaptation

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tox/367/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tox/367/console

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tox/368/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tox/368/console

Plug to branch

Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tox/369/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tox/369/console

Build is green See https://jenkins.softwareheritage.org/job/DCORE/job/tox/372/ for more details.

! In !113 (closed), @swh-public-ci wrote: Build has FAILED

Link to build: https://jenkins.softwareheritage.org/job/DCORE/job/tox/369/ See console output for more information: https://jenkins.softwareheritage.org/job/DCORE/job/tox/369/console

Quoting me on the diff's description for that.

Note: I expected the ci build to fail. One test needs the "optional" lzip dependency installed.

And it did.

Fixed the ci build it by adding the lzip dependency on ci node.

Note: This means that we will need to add the lzip runtime dependency on a debian package somewhere (either swh-core's as it's the one exposing it or the client of this, loader-core comes to mind).

Merge request was accepted

approved this merge request

Some references in the commit message have been migrated:

• T2110 is now swh-loader-core#2110 (closed)

Rebase and plug to master

Build is green See https://jenkins.softwareheritage.org/job/DCORE/job/tox/381/ for more details.