Migrate jenkins nodes to bullseye
This means following the plan [3] for:
- jenkins-debian1: the debian package build node [1]
- thyssen: jenkins server node [2]
It should be fine to just upgrade the nodes as their respective major tooling are part of debian packages.
-
[1] The main tools here are sbuild and chroot.
-
[2] The main tooling are within docker constraint (so it's "pinned")
-
[3]
task=#3770
puppet agent --disable "$task: dist-upgrade to bullseye"
cd /etc
sed -i -e 's/buster/bullseye/' /etc/apt/sources.list.d/*
sed -i -e 's,bullseye/updates,bullseye-security,' /etc/apt/sources.list.d/debian-security.list
git status
grep bullseye-security /etc/apt/sources.list.d/debian-security.list
git add .
git commit -m "$task: Migrate sources.list to bullseye"
apt update
apt upgrade -y
apt dist-upgrade -y
reboot
puppet agent --enable && puppet agent --testMigrated from T3770 (view on Phabricator)
Activity
mentioned in issue #3579 (closed)
marked this issue as related to #3579 (closed)
added Component upgrades priority:Normal labels
mentioned in commit swh/infra/puppet/puppet-swh-site@68c4aae2
mentioned in commit swh/infra/puppet/puppet-swh-site@006aefad
mentioned in commit swh/infra/puppet/puppet-environment@7e985f56
added state:wip label
mentioned in commit swh/infra/puppet/puppet-swh-site@f7b31905
! In #3770 (closed), @ardumont wrote: As amended in the description, jenkins-debian1 is done. I'm waiting for the current swh-web release to come through to ensure there is no actual side-effect.
That build failed somewhat unrelatedly to the migration. Nothing is broken. It failed because there was a
debian/in the master branch's.gitignorefile. And the now new git version (from bullseye) is be more strict. It refused justly to comply. The fix is landed (in master, drop that wrong instruction, then release). I'm pretty sure it's fine now (build still ongoing).As swh-web's build si too slow, i triggered another (swh.loader.core) and it went fine [1]
Then, i'll attend to thyssen.
So, no more blocker besides people using the ci ;)
mentioned in commit swh/infra/ci-cd/swh-jenkins-jobs@5485cc04
-
Heads up, this ui should be pretty cool to prevent build from happening prior to the actual dist-upgrade [1].
-
Thyssen:
- Prepare jenkins in "prepare shutdown" given this task as notice (using the previous link)
- Upgrade according to the plan
- Rebooted
So thyssen is now in bullseye.
ardumont@thyssen:~% lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 11 (bullseye) Release: 11 Codename: bullseye ardumont@thyssen:~% uptime 07:38:13 up 0 min, 2 users, load average: 2.98, 0.74, 0.25Remains to check that everything is fine.
-
unbound was unhappy though [1]
The configuration file (apparently installed manually) had to change:
root@thyssen:~# cat /etc/unbound/unbound.conf.d/listen-docker.conf interface: 0.0.0.0 interface: ::0 access-control: 127.0.0.0/8 allow access-control: 172.16.0.0/12 allowa access-control: 192.168.0.0/16 allow root@thyssen:~# unbound-checkconf /etc/unbound/unbound.conf.d/listen-docker.conf:1: error: syntax error read /etc/unbound/unbound.conf failed: 1 errors in configuration file root@thyssen:~# unbound -dd /etc/unbound/unbound.conf.d/listen-docker.conf:1: error: syntax error read /etc/unbound/unbound.conf failed: 1 errors in configuration file [1638952385] unbound[6720:0] fatal error: Could not read config file: /etc/unbound/unbound.conf. Maybe try unbound -dd, it stays on the commandline to see more errors, or unbound-checkconfThe correct syntax becomes:
root@thyssen:~# cat /etc/unbound/unbound.conf.d/listen-docker.conf server: interface: 0.0.0.0 interface: ::0 access-control: 127.0.0.0/8 allow access-control: 172.16.0.0/12 allow access-control: 192.168.0.0/16 allow root@thyssen:~# unbound-checkconf unbound-checkconf: no errors in /etc/unbound/unbound.conf root@thyssen:~# unbound -dd [1638952639] unbound[8445:0] error: cannot open pidfile /run/unbound.pid: Permission denied [1638952639] unbound[8445:0] notice: init module 0: subnet [1638952639] unbound[8445:0] notice: init module 1: validator [1638952639] unbound[8445:0] notice: init module 2: iterator [1638952639] unbound[8445:0] info: start of service (unbound 1.13.1). [1638952646] unbound[8445:0] info: generate keytag query _ta-4f66. NULL IN ^C[1638952658] unbound[8445:0] info: service stopped (unbound 1.13.1). [1638952658] unbound[8445:0] info: server stats for thread 0: 12 queries, 0 answers from cache, 12 recursions, 0 prefetch, 0 rejected by ip ratelimiting [1638952658] unbound[8445:0] info: server stats for thread 0: requestlist max 1 avg 0.5 exceeded 0 jostled 0 [1638952658] unbound[8445:0] info: average recursion processing time 0.169166 sec [1638952658] unbound[8445:0] info: histogram of recursion processing times [1638952658] unbound[8445:0] info: [25%]=0.0229376 median[50%]=0.032768 [75%]=0.218453 [1638952658] unbound[8445:0] info: lower(secs) upper(secs) recursions [1638952658] unbound[8445:0] info: 0.000512 0.001024 1 [1638952658] unbound[8445:0] info: 0.016384 0.032768 5 [1638952658] unbound[8445:0] info: 0.065536 0.131072 1 [1638952658] unbound[8445:0] info: 0.131072 0.262144 3 [1638952658] unbound[8445:0] info: 0.524288 1.000000 2 CTRL-C root@thyssen:~# systemctl restart unbound root@thyssen:~# systemctl status unbound | head -3 ● unbound.service - Unbound DNS server Loaded: loaded (/lib/systemd/system/unbound.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-12-08 08:38:35 UTC; 4min 59s ago- [1] $1229
I need to check with the team how to properly do that (well how we do it).
Well, just check all plugins and trigger update.
-
I need to check with the team how to properly do that (well how we do it).
Well, just check all plugins and trigger update.
Plugin upgrade done through the jenkins ui. Then restart jenkins.
root@thyssen# systemctl restart jenkinsJenkins is now up-to-date is running new builds. Everything is fine.
This task can be closed.
assigned to @ardumont
removed state:wip label