Upgrade ArgoCD to 2.10.1

To do #5248 (closed) in a DRY way, I would like to use a newer version of the ArgoCD ApplicationSet resource, which supports full go templating.

To do so, we need to upgrade ArgoCD to the 2.10 branch.

Checklist:

• Check upstream changelogs
  • 2.8 to 2.9 - https://argo-cd.readthedocs.io/en/stable/operator-manual/upgrading/2.8-2.9/
  • 2.9 to 2.10 - https://argo-cd.readthedocs.io/en/stable/operator-manual/upgrading/2.9-2.10/
• Check 2.10.1 known issues
• Backup argocd
• Disable AutoSync on all resources
• Bump the ArgoCD resource to v2.10.1
• Run the upgrade
• Fix the fallout
• Revert AutoSync change

changed milestone to %Dynamic infrastructure [Roadmap - Tooling and infrastructure]

assigned to @olasd

changed the description

No changes of note between 2.8 and 2.10, only version upgrades of built-in tools that shouldn't affect us.

marked the checklist item Check upstream changelogs as completed

marked the checklist item Backup argocd as completed

ArgoCD backup:

kubectl --context cluster-admin-rke2 exec -it -n argocd argocd-server-7584db7ccf-fjf79 -- argocd admin export > argocd-export.yaml

marked the checklist item Disable AutoSync on all resources as completed

To disable autosync on all resources, I've created a manual sync window on the default project denying autosync.

mentioned in commit swh/infra/ci-cd/k8s-clusters-conf@1eff1a6c

marked the checklist item Bump the ArgoCD resource to v2.10.1 as completed

marked the checklist item Run the upgrade as completed

Only fallout to report : the two configmaps creating our users were erased by the upgrade, so they had to be applied again.

The helm chart (instead of the plain manifest as generated by kustomize) supports extending the contents of the configmaps, so I updated our admin-rke2-argocd-install app to use it. Looks like using argocd itself to migrate from one to the other didn't work :-)

mentioned in commit swh/infra/ci-cd/k8s-clusters-conf@fd43a545

marked the checklist item Fix the fallout as completed

After cleaning up the existing resources for the argocd deployment, and manually installing the helm chart, ArgoCD could redeploy itself and everything is in sync.

I'll turn off the deny sync window and argocd will be back to normal!

marked the checklist item Revert AutoSync change as completed

closed