Update Dulwich to v0.21.5

In order to resolve swh/devel/swh-loader-git#4749

mentioned in merge request swh/devel/swh-loader-git!154 (merged)

Doing the Debian packaging for this.

assigned to @olasd

In swh-apps, the following containers use dulwich 0.21.3 and need an update:

• swh-loader-git
• swh-loader-bzr
• swh-scrubber

(I guess only swh-loader-git really does need an update, and the others can wait for an incidental update)

(I guess only swh-loader-git really does need an update, and the others can wait for an incidental update)

indeed

Recent dulwich packages want a newer urllib3 (>= 1.25) that's in buster(-backports) (1.24).

Looks like the point is getting urllib3's new default certificate validation behavior https://urllib3.readthedocs.io/en/latest/user-guide.html#certificate-verification.

As I'm pretty sure we really want to turn that off anyway, I'll relax the dependency and see how things go.

Dulwich 0.21.5 packages built and uploaded to the swh debian repository (for unstable-swh and buster-swh).

mentioned in commit swh-apps@b578ab30

Updated swh-loader-git container image: 20230530.2

mentioned in commit swh/infra/ci-cd/swh-charts@b782dcee

mentioned in merge request swh/infra/ci-cd/swh-charts!58 (merged)

mentioned in commit swh/infra/ci-cd/swh-charts@2407f16c

The new image is deployed to staging...

So, looks like I've generated a (very) broken dulwich package for buster-swh.

Turns out dulwich now depends on a setuptools that know what pyproject.toml is (that is, setuptools >=61). buster-swh does not have such a setuptools version.

Backporting setuptools 61.x to buster doesn't seem to work (failure in a debian patch). So I guess I'll just move the pyproject.toml stuff back into setup.cfg in a local patch in our buster-swh package. Meh.

Updated the dulwich 0.21.5 package for buster-swh to actually contain dulwich, things are now running okay on worker01.

dulwich 0.21.5 deploying to all production git loaders now, should be done in 20 mins.

closed