Meta-task: upgrade infrastructure to Debian Bullseye

Plan:

Upgrade the node to bullseye
Update the inventory [1] once the migration is done to mention the new distribution
Update Vagrantfile to use the corresponding debian 11 box
[1] https://inventory.internal.softwareheritage.org/

Servers to upgrade

#3771 (closed) Proxmox hypervisors
- beaubourg.softwareheritage.org
- branly.internal.softwareheritage.org
- hypervisor3.internal.softwareheritage.org
- pompidou.internal.softwareheritage.org
- uffizi.softwareheritage.org

(!) Proxmox/Ceph upgrade ?

Delayed migrations

//Need some actions to separate swh services from the raw storage or a bullseye version of the swh packages//

counters
- counters0.internal.staging.swh.network
- counters1.internal.softwareheritage.org redis needs storage, we could separate the search services / journal client from redis
Object storage / Storage
- saam.internal.softwareheritage.org
- storage1.internal.staging.swh.network
- banco.softwareheritage.org need to generate a swh-storage package for bullseye / don't use the debian package deployment need to generate an obj-storage package for bullseye / don't use the debian package deployment checkup backup compatibility
graph
- granet.internal.softwareheritage.org need to check the dependencies / installed packages (-> one is python3-swh.graph.client) possibility to deploy the graph service dynamically ?
getty.internal.softwareheritage.org
- used as a sandbox
- CMAK compatibility ?
scheduler
- scheduler0.internal.staging.swh.network
- saatchi.internal.softwareheritage.org Need to separate rabbitmq from the swh services and journal clients

(!) rabbitmq version has changed, 3.7.8 in debian10, 3.8.9 in debian 11. The puppet module seems broken due to wrong dependencies when installing rabbitmqadmin

tate.softwareheritage.org phabricator should be migrated to gitlab mediawiki should be migrated jenkins and keycloak RP should be kept / migrated

Servers left in busters

//services will be migrated to a new deployment system//

Services to upgrade

build
- ci images
- base swh image
debian packages
- librdkafka ?
- monitoring-plugins-systemd
- prometheus-statsd-exporter
- python3-elasticsearch
- python3-amqp
- python3-confluent-kafka
- python3-importlib-metadata
- python3-kombu

Desktop

We need to double check with their owner for the upgrade whether they want to do it or not (questions asked on irc, awaiting response)

#3716 (closed): grouss: giverny.softwareheritage.org (currently from jessie to buster)
seirl: grand-palais.softwareheritage.org (migrated to bullseye)

Firewalls

#3706 (closed), #3760 (closed) minor then major opnsense upgrade (not bullseye migration)
- glyptotek.internal.softwareheritage.org (bsd)
- pushkin.internal.softwareheritage.org (bsd)

Decommission

swh/infra/sysadm-environment#3792 (closed): louvre.internal.softwareheritage.org
swh/infra/sysadm-environment#3805 (closed): boatbucket.internal.softwareheritage.org
journal0.internal.staging.swh.network
dbreplica0.euwest.azure.internal.softwareheritage.org (failing main db replica, cf. swh/infra/sysadm-environment#2581)
dbreplica1.euwest.azure.internal.softwareheritage.org (indexer replica db, pg 11)

Unneeded migration

mirror-test.internal.staging.swh.network

node used by douardda, no swh packages are installed on it Reason:

Tue 14 Dec 2021 06:09:57 PM CET
16:59 <+ardumont> douardda: btw, do you still need mirror-test node, can we migrate it or shall we drop it out of the migration path (#3579)?
17:00 <+douardda> I won't use it any time soon, so you can drop it

Migrated from T3579 (view on Phabricator)

35 of 62 checklist items completed · Edited 2 years ago

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information