Install the 2 new bare metal servers for cassandra production cluster

Orders: https://mybox.inria.fr/smart-link/13004275-5cee-433b-ba88-3c73141757f0/

cassandra09: Inventory: https://inventory.internal.admin.swh.network/dcim/devices/262/

Summary:

• Management address (DNS): 128.93.134.47 (swh-cas9-adm.inria.fr)
• VLAN configuration: VLAN440
• Internal IP(s): 192.168.100.189
• Internal DNS name(s): cassandra09.internal.softwareheritage.org

cassandra10: Inventory: https://inventory.internal.admin.swh.network/dcim/devices/263/

Summary:

• Management address (DNS): 128.93.134.48 (swh-cas10-adm.inria.fr)
• VLAN configuration: VLAN440
• Internal IP(s): 192.168.100.190
• Internal DNS name(s): cassandra10.internal.softwareheritage.org

Tasks:

• Declare the servers in the inventory
• Add the management info in the credential store
• Install the OS
• Add puppet configuration (if needed)
• Configure firewall rules (if needed)
• Configure zfs (only one directory for data and commitlog mountpoint => /srv/cassandra/instance1)
• Add the servers one by one in the cluster
  • Launch a full repair on one node and wait for the end
  • Launch a full repair on the other node and wait for the end
  • Launch a cleanup on all the servers except the last one:
    • cassandra01
    • cassandra02
    • cassandra03
    • cassandra04
    • cassandra05
    • cassandra06
    • cassandra07
    • cassandra08
    • cassandra09
• Declare the servers in the prometheus rules to activate the metrics and monitoring

changed milestone to %Cassandra in production as primary storage [Roadmap - Tooling and infrastructure]

added cassandra hardware labels

assigned to @vsellier

changed the description

taking the issue until the servers are racked

changed the description

marked the checklist item Declare the servers in the inventory as completed

marked the checklist item Add the management info in the credential store as completed

mentioned in commit ipxe@138f3b5e

mentioned in commit swh/infra/puppet/puppet-swh-site@72c86cf4

mentioned in commit swh/infra/puppet/puppet-swh-site@174cc3b6

mentioned in commit ipxe@ebdebb65

mentioned in commit swh/infra/puppet/puppet-swh-site@b5c6f1e5

marked the checklist item Install the OS as completed

marked the checklist item Add puppet configuration (if needed) as completed

marked the checklist item Configure zfs (only one directory for data and commitlog mountpoint => /srv/cassandra/instance1) as completed

mentioned in commit swh/infra/ci-cd/k8s-clusters-conf@f95217e5

mentioned in merge request swh/infra/ci-cd/k8s-clusters-conf!26 (merged)