Skip to content

Split up pergamon to smaller VMs

Pergamon is actually a VM running on the louvre hypervisor.

Since about a week, it has been experiencing various performance issues, including but not limited to higher loads, increased amounts of I/O wait time and completely locked up virtual CPUs:

kernel:[259911.781535] NMI watchdog: BUG: soft lockup - CPU#3 stuck for 134s! [systemd-cgroups:666]

This VM is running a huge amount of services. Among them, we have:

  • a DNS server for the internal softwareheritage networks
  • Icinga monitoring
  • Prometheus monitoring
  • Munin
  • Puppet
  • A Debian repository

This is a bit much for a single VM. Creating smaller ones dedicated to one or two service at most would help isolate the services and avoid performance impacts among them. If a service were to require more hardware resources than the others, its VM could be migrated to a less loaded/more powerful hypervisor.

Migrated from T1166 (view on Phabricator)