admin: Assess pros/cons of migrating static services to kubernetes

This only deals with the admin environment.

• Goals
• Impacts
• Assessment plan
• Services
• Existing

Goals

Keep up with upstream upgrades. We currently cannot (for example netbox is late ¹)

Upgrade should be transparent (without downtime). There is currently no rolling upgrade on most of our services (that comes defacto with kubernetes).

Impacts

• Charts development
• Data migration from previous backends to the ones managed in kubernetes

Assessment plan

• make a list of the tools/services deployed as VMs (next chapter ²)
• for each tool/service
  • look into whether container images, helm chart, etc... exist;
  • sketch what the kube deployment would look like;
  • make a plan for migrating the data into the kube deployment (manual restore of the db, I don't know what);
  • then compare that work with current puppet-based maintenance is ok

Services

List of tools/services deployed per vm in the admin realm.

|---------------+---------------------+------------------------+----------------|
| Service       | Purpose             | Current running on vms | Clients        |
|---------------+---------------------+------------------------+----------------|
| netbox        | hardware inventory  | bojimans.admin         | sysop          |
| keycloak      | authentication      | kelvingrove.admin      | swh services   |
| reverse-proxy | //                  | rp1.admin              | swh services   |
| thanos        | //                  | thanos.admin           | swh services   |
| dbs           | postgresql servers  | dali.admin             | sysop services |
| sentry        | swh services issues | riverside.admin        | swh services   |
| grafana       | monitoring board    | grafana0.admin         | swh services   |
|---------------+---------------------+------------------------+----------------|

Existing

• Postgresql db can be managed in kube (but it's not completely assessed either).

1. #5412 ↩

2. #5602 ↩

changed the description

changed the description

For the Keycloak upgrade, I think the simplest way would be to export our current realms and users to JSON and import them back into latest Keycloak version.

Within our docker environment I could export those data by executing this command into the keycloak service (using Keycloak 10.0.2):

$ /opt/jboss/keycloak/bin/standalone.sh -Djboss.socket.binding.port-offset=100 -Dkeycloak.migration.action=export -Dkeycloak.migration.provider=singleFile -Dkeycloak.migration.realmName=SoftwareHeritage -Dkeycloak.migration.usersExportStrategy=REALM_FILE -Dkeycloak.migration.file=/tmp/export.json

Then I installed Keycloak 26.1.3 locally and could successfully import the SoftwareHeritage realm and its users from the admin console.

There is also the custom SWH themes to update but that is another story.

I did more analysis on the Keycloak migration:

• users can be exported into multiple files for better import performances afterwards
• while Keycloak can export users, it cannot export their offline sessions meaning all Web API tokens generated with the previous instance are no longer valid with the new instance

So the main drawback of migrating using export files is that all previous Web API tokens will become invalid, meaning we will have to inform users about the fact they will need to generate a new token once the migration effective.

I upgraded the keycloak service in our docker environment to Keycloak 26.1.3 and authentication related features are still working as expected.

Regarding the custom SWH themes to update, I found the keycloakify tool that should help a lot in that process.

mentioned in issue #5603

changed title from Assess admin network services migration to kubernetes to admin: Assess remaining services migration to kubernetes

changed title from admin: Assess remaining services migration to kubernetes to admin: Assess pros/cons of migrating static services to kubernetes

changed the description

mentioned in issue #5604

mentioned in issue #5415 (closed)