Skip to content

Meta-task: upgrade infrastructure to Debian Bullseye

Plan:

Servers to upgrade

  • #3771 (closed) Proxmox hypervisors
    • beaubourg.softwareheritage.org
    • branly.internal.softwareheritage.org
    • hypervisor3.internal.softwareheritage.org
    • pompidou.internal.softwareheritage.org
    • uffizi.softwareheritage.org

(!) Proxmox/Ceph upgrade ?

  • #3761 (closed): riverside.internal.softwareheritage.org (sentry)

  • #3762 (closed): kafka[1-4].internal.staging.swh.network

  • #3751 (closed): cluster log

    • esnode[1-3].internal.softwareheritage.org
    • kibana0.internal.softwareheritage.org
    • logstash0.internal.softwareheritage.org (ELK stack upgrade to 7.15.2)

  • #3751 (closed): search elasticsearch

    • search-esnode[4-6].internal.softwareheritage.org
    • esnode0.internal.staging.swh.network

  • #3801 (closed): Postgresql databases

    • belvedere.internal.softwareheritage.org
    • db1.internal.staging.swh.network
    • somerset.internal.softwareheritage.org

  • #3749 (closed): hedgedoc.softwareheritage.org (bardo.internal.admin.swh.network)

  • #3802 (closed): bojimans.internal.softwareheritage.org (netbox)

  • #3807 (closed): kelvingrove.internal.softwareheritage.org (keycloak)

  • Admin / DNS

    • #3800 (closed): ns0.euwest.azure.internal.softwareheritage.org
    • #3808: pergamon.softwareheritage.org ((!): check the plan in the task)

  • #3770 (closed): CI servers

    • jenkins-debian1.internal.softwareheritage.org
    • thyssen.internal.softwareheritage.org

Delayed migrations

//Need some actions to separate swh services from the raw storage or a bullseye version of the swh packages//

  • counters

    • counters0.internal.staging.swh.network
    • counters1.internal.softwareheritage.org redis needs storage, we could separate the search services / journal client from redis

  • Object storage / Storage

    • saam.internal.softwareheritage.org
    • storage1.internal.staging.swh.network
    • banco.softwareheritage.org need to generate a swh-storage package for bullseye / don't use the debian package deployment need to generate an obj-storage package for bullseye / don't use the debian package deployment checkup backup compatibility

  • graph

    • granet.internal.softwareheritage.org need to check the dependencies / installed packages (-> one is python3-swh.graph.client) possibility to deploy the graph service dynamically ?

  • getty.internal.softwareheritage.org

    • used as a sandbox
    • CMAK compatibility ?

  • scheduler

    • scheduler0.internal.staging.swh.network
    • saatchi.internal.softwareheritage.org Need to separate rabbitmq from the swh services and journal clients

(!) rabbitmq version has changed, 3.7.8 in debian10, 3.8.9 in debian 11. The puppet module seems broken due to wrong dependencies when installing rabbitmqadmin

  • tate.softwareheritage.org phabricator should be migrated to gitlab mediawiki should be migrated jenkins and keycloak RP should be kept / migrated

Servers left in busters

//services will be migrated to a new deployment system//

  • worker[0-3].internal.staging.swh.network
  • worker[01-13].euwest.azure.internal.softwareheritage.org
  • worker[01-17].softwareheritage.org
  • search0.internal.staging.swh.network
  • search1.internal.softwareheritage.org
  • deposit.internal.staging.swh.network
  • vault.internal.staging.swh.network
  • vangogh.euwest.azure.internal.softwareheritage.org
  • webapp.internal.staging.swh.network
  • webapp1.internal.softwareheritage.org
  • moma.softwareheritage.org
  • #3799 (closed): rp0.internal.staging.swh.network
  • rp1.internal.admin.swh.network
  • objstorage0.internal.staging.swh.network expose the objstorage service to the mirrors
  • storage01.euwest.azure.internal.softwareheritage.org r/o storage

Services to upgrade

  • build
    • ci images
    • base swh image
  • debian packages
    • librdkafka ?
    • monitoring-plugins-systemd
    • prometheus-statsd-exporter
    • python3-elasticsearch
    • python3-amqp
    • python3-confluent-kafka
    • python3-importlib-metadata
    • python3-kombu

Desktop

We need to double check with their owner for the upgrade whether they want to do it or not (questions asked on irc, awaiting response)

  • #3716 (closed): grouss: giverny.softwareheritage.org (currently from jessie to buster)
  • seirl: grand-palais.softwareheritage.org (migrated to bullseye)

Firewalls

  • #3706 (closed), #3760 (closed) minor then major opnsense upgrade (not bullseye migration)
    • glyptotek.internal.softwareheritage.org (bsd)
    • pushkin.internal.softwareheritage.org (bsd)

Decommission

Unneeded migration

  • mirror-test.internal.staging.swh.network

node used by douardda, no swh packages are installed on it Reason:

Tue 14 Dec 2021 06:09:57 PM CET
16:59 <+ardumont> douardda: btw, do you still need mirror-test node, can we migrate it or shall we drop it out of the migration path (#3579)?
17:00 <+douardda> I won't use it any time soon, so you can drop it

Migrated from T3579 (view on Phabricator)

Edited by Phabricator Migration user
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information