swh: Allow centralized pv/pvcs declaration

This allows to share the pv/pvcs declaration instead of duplicating it inside the various templates using a duplicated declaration (e.g. graph, alter, cassandra-checks, ...).

It's best to review the mr commit per commit:

• 1 commit allows pvs and pvcs declaration
• 1 commit reworks the graph deployment to use pvs/pvcs declaration [1]
• the alter commit aligns the alter deployment to pvs/pvcs declaration [2]
• 1 commit drops the unused cassandra pvc declaration check which was not used.
• 1 commit adapts the objstorage deployment to add a new objstorage-ro rpc instance which now shares the newly declared volume between the objstorage rpc rw and the rpc ro instances. [3]
• 1 commit does does the same for the local cluster. This got tested to work in the local-cluster with a similar setup (commit included) [4]
• At last, 1 commit in the cluster-components which allows to declare volumes as well. It's then moving the objstorage-pathslicing-pvc declared in the next-version namespace in the cluster-components. That way, the pvc won't get decommissionned when the argocd next-version app is closed (after the chart mr is merged). The pv associated is thus no longer resetted (which is the case currently) [5]

Note: The diff should be null for the pvs/pvcs changes but as the toYaml somehow generates different order the pv/pvcs, the diffs are minimalistics (some entries are shuffled around but the values remains the same).

[1] helm diff on graph deployment

[swh] Comparing changes between branches production and mr/centralize-pv-pvcs (per environment)...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...


------------- diff for environment staging namespace swh -------------

No differences


------------- diff for environment staging namespace swh-cassandra -------------

--- /tmp/swh-chart.swh.urjwplFS/staging-swh-cassandra.before	2025-03-07 12:01:59.920857760 +0100
+++ /tmp/swh-chart.swh.urjwplFS/staging-swh-cassandra.after	2025-03-07 12:02:00.464840181 +0100
@@ -9859,38 +9859,38 @@
   namespace: swh-cassandra
 spec:
   storageClassName: cephfs
   accessModes:
     - ReadWriteMany
   volumeMode: Filesystem
   resources:
     requests:
       storage: 30Gi
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-python3k-inmemory-pvc
   namespace: swh-cassandra
   labels:
     app: graph-grpc-python3k
 spec:
   accessModes:
   - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi
   storageClassName: local-path
   volumeMode: Filesystem
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-python3k-persistent-pvc
   namespace: swh-cassandra
   labels:
     app: graph-grpc-python3k
 spec:
   accessModes:
   - ReadWriteOnce


------------- diff for environment staging namespace next-version -------------

--- /tmp/swh-chart.swh.urjwplFS/staging-next-version.before	2025-03-07 12:02:00.048853625 +0100
+++ /tmp/swh-chart.swh.urjwplFS/staging-next-version.after	2025-03-07 12:02:00.592836045 +0100
@@ -4600,38 +4600,38 @@
         auth_token: ${SVIX_AUTH_TOKEN}
         server_url: https://svix-next-version.internal.staging.swh.network
     journal:
       brokers:
         - kafka-cluster-kafka-brokers:9092
       auto_offset_reset: latest
       group_id: staging-next-version-archive-webhooks
       object_types:
       - origin_visit_status
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: swh-graph-grpc-dataset-example-pvc
   namespace: swh-cassandra-next-version
   labels:
     app: graph-grpc-example
 spec:
   accessModes:
   - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi
   storageClassName: local-persistent
   volumeMode: Filesystem
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: swh-graph-grpc-inmemory-pvc
   namespace: swh-cassandra-next-version
   labels:
     app: graph-grpc-example
 spec:
   accessModes:
   - ReadWriteOnce


------------- diff for environment production namespace swh -------------

No differences


------------- diff for environment production namespace swh-cassandra -------------

--- /tmp/swh-chart.swh.urjwplFS/production-swh-cassandra.before	2025-03-07 12:02:01.068820664 +0100
+++ /tmp/swh-chart.swh.urjwplFS/production-swh-cassandra.after	2025-03-07 12:02:01.540805413 +0100
@@ -11595,92 +11595,94 @@
       brokers:
         - kafka1.internal.softwareheritage.org:9092
         - kafka2.internal.softwareheritage.org:9092
         - kafka3.internal.softwareheritage.org:9092
         - kafka4.internal.softwareheritage.org:9092
       auto_offset_reset: latest
       group_id: swh-archive-prod-webhooks
       object_types:
       - origin_visit_status
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume.yaml
 apiVersion: v1
 kind: PersistentVolume
 metadata:
   name: graph-20241206-persistent-local-pv
   namespace: swh-cassandra
 spec:
-  capacity:
-    storage: 1Gi
-  volumeMode: Filesystem
   accessModes:
   - ReadWriteOnce
-  persistentVolumeReclaimPolicy: Retain
-  storageClassName: local-storage
+  capacity:
+    storage: 1Gi
   local:
     path: /srv/softwareheritage/ssd/graph/2024-12-06/compressed
   nodeAffinity:
     required:
       nodeSelectorTerms:
       - matchExpressions:
         - key: kubernetes.io/hostname
           operator: In
-          values: 
-            - rancher-node-highmem01
+          values:
+          - rancher-node-highmem01
+  persistentVolumeReclaimPolicy: Retain
+  storageClassName: local-storage
+  volumeMode: Filesystem
 ---
 # Source: swh/templates/alter/deployment.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: alter-recovery-bundles-pvc
   namespace: swh-cassandra
 spec:
   storageClassName: cephfs
   accessModes:
     - ReadWriteMany
   volumeMode: Filesystem
   resources:
     requests:
       storage: 30Gi
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-20241206-inmemory-pvc
   namespace: swh-cassandra
   labels:
     app: graph-grpc-20241206
 spec:
   accessModes:
   - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi
   storageClassName: local-path
   volumeMode: Filesystem
 ---
-# Source: swh/templates/graph/persistent-volume-claims.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-20241206-persistent-pvc
   namespace: swh-cassandra
+  labels:
+    app: graph-grpc-20241206
 spec:
+  accessModes:
+  - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi
+  storageClassName: local-storage
   volumeMode: Filesystem
   volumeName: graph-20241206-persistent-local-pv
-  storageClassName: local-storage
-  accessModes:
-  - ReadWriteOnce
 ---
 # Source: swh/templates/counters/rpc-service.yaml
 apiVersion: v1
 kind: Service
 metadata:
   name: counters-rpc
   namespace: swh-cassandra
   labels:
     app: counters-rpc
 spec:

[2] helm diff on alter deployment

[swh] Comparing changes between branches production and mr/centralize-pv-pvcs (per environment)...
Your branch is ahead of 'origin/production' by 5 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
Your branch is ahead of 'origin/production' by 5 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...


------------- diff for environment staging namespace swh -------------

No differences


------------- diff for environment staging namespace swh-cassandra -------------

--- /tmp/swh-chart.swh.WMKOZRCT/staging-swh-cassandra.before	2025-03-07 12:28:01.318374812 +0100
+++ /tmp/swh-chart.swh.WMKOZRCT/staging-swh-cassandra.after	2025-03-07 12:28:01.878356693 +0100
@@ -9844,34 +9844,36 @@
         auth_token: ${SVIX_AUTH_TOKEN}
         server_url: https://svix.internal.staging.swh.network
     journal:
       brokers:
         - journal2.internal.staging.swh.network:9092
       auto_offset_reset: latest
       group_id: swh-archive-stg-webhooks
       object_types:
       - origin_visit_status
 ---
-# Source: swh/templates/alter/deployment.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: alter-recovery-bundles-pvc
   namespace: swh-cassandra
+  labels:
+    app: alter
 spec:
-  storageClassName: cephfs
   accessModes:
-    - ReadWriteMany
-  volumeMode: Filesystem
+  - ReadWriteMany
   resources:
     requests:
       storage: 30Gi
+  storageClassName: cephfs
+  volumeMode: Filesystem
 ---
 # Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-python3k-inmemory-pvc
   namespace: swh-cassandra
   labels:
     app: graph-grpc-python3k
 spec:


------------- diff for environment staging namespace next-version -------------

No differences


------------- diff for environment production namespace swh -------------

No differences


------------- diff for environment production namespace swh-cassandra -------------

--- /tmp/swh-chart.swh.WMKOZRCT/production-swh-cassandra.before	2025-03-07 12:28:02.482337151 +0100
+++ /tmp/swh-chart.swh.WMKOZRCT/production-swh-cassandra.after	2025-03-07 12:28:02.954321880 +0100
@@ -11622,34 +11622,36 @@
       nodeSelectorTerms:
       - matchExpressions:
         - key: kubernetes.io/hostname
           operator: In
           values:
           - rancher-node-highmem01
   persistentVolumeReclaimPolicy: Retain
   storageClassName: local-storage
   volumeMode: Filesystem
 ---
-# Source: swh/templates/alter/deployment.yaml
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: alter-recovery-bundles-pvc
   namespace: swh-cassandra
+  labels:
+    app: alter
 spec:
-  storageClassName: cephfs
   accessModes:
-    - ReadWriteMany
-  volumeMode: Filesystem
+  - ReadWriteMany
   resources:
     requests:
       storage: 30Gi
+  storageClassName: cephfs
+  volumeMode: Filesystem
 ---
 # Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
   name: graph-20241206-inmemory-pvc
   namespace: swh-cassandra
   labels:
     app: graph-grpc-20241206
 spec:

[3] helm diff to evolve the next-version/local-cluster objstorage

[swh] Comparing changes between branches production and mr/centralize-pv-pvcs (per environment)...
Your branch is ahead of 'origin/production' by 6 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
Your branch is ahead of 'origin/production' by 6 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...


------------- diff for environment staging namespace swh -------------

No differences


------------- diff for environment staging namespace swh-cassandra -------------

No differences


------------- diff for environment staging namespace next-version -------------

--- /tmp/swh-chart.swh.Q0zchbWG/staging-next-version.before	2025-03-07 13:56:16.446480298 +0100
+++ /tmp/swh-chart.swh.Q0zchbWG/staging-next-version.after	2025-03-07 13:56:17.098459168 +0100
@@ -1877,24 +1877,107 @@
     root:
       level: "INFO"
       handlers:
       - console
 ---
 # Source: swh/templates/objstorage/configmap.yaml
 apiVersion: v1
 kind: ConfigMap
 metadata:
   namespace: swh-cassandra-next-version
+  name: objstorage-ro-pathslicing-configuration-template
+data:
+  config.yml.template: |
+    objstorage:
+      client_max_size: 1073741824
+      cls: pathslicing
+      root: /srv/swh/objects
+      slicing: 0:1/1:5
+---
+# Source: swh/templates/objstorage/configmap.yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  namespace: swh-cassandra-next-version
+  name: objstorage-ro-pathslicing-configuration-logging
+data:
+  logging-gunicorn.json: |
+    {
+      "disable_existing_loggers": true,
+      "filters": {
+        "throttle_accesslog": {
+          "interval": 60,
+          "status_codes": [
+            200,
+            400
+          ]
+        }
+      },
+      "formatters": {
+        "default": {
+          "class": "pythonjsonlogger.jsonlogger.JsonFormatter",
+          "format": "[%(asctime)s: %(levelname)s/%(processName)s-%(process)d %(threadName)s(%(pathname)s:%(lineno)s:%(funcName)s)]%(name)s %(message)s"
+        }
+      },
+      "handlers": {
+        "console": {
+          "class": "logging.StreamHandler",
+          "formatter": "default",
+          "level": "INFO",
+          "stream": "ext://sys.stdout"
+        }
+      },
+      "loggers": {
+        "azure": {
+          "level": "WARN"
+        },
+        "gunicorn": {
+          "level": "INFO"
+        },
+        "gunicorn.access": {
+          "filters": [
+            "throttle_accesslog"
+          ],
+          "handlers": [
+            "console"
+          ],
+          "level": "WARN"
+        },
+        "gunicorn.error": {
+          "level": "INFO"
+        },
+        "swh": {
+          "level": "INFO"
+        },
+        "swh.core.statsd": {
+          "level": "INFO"
+        }
+      },
+      "root": {
+        "handlers": [
+          "console"
+        ],
+        "level": "INFO"
+      },
+      "version": 1
+    }
+---
+# Source: swh/templates/objstorage/configmap.yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  namespace: swh-cassandra-next-version
   name: objstorage-rw-pathslicing-configuration-template
 data:
   config.yml.template: |
     objstorage:
+      allow_delete: true
       client_max_size: 1073741824
       cls: pathslicing
       root: /srv/swh/objects
       slicing: 0:1/1:5
 ---
 # Source: swh/templates/objstorage/configmap.yaml
 apiVersion: v1
 kind: ConfigMap
 metadata:
   namespace: swh-cassandra-next-version
@@ -2440,21 +2523,21 @@
         password: ${CASSANDRA_RW_PASSWORD}
         username: ${CASSANDRA_RW_USERNAME}
       cls: cassandra
       consistency_level: LOCAL_QUORUM
       hosts:
       - cassandra-cluster-next-version-service
       keyspace: swh
       objstorage:
         cls: remote
         enable_requests_retry: true
-        url: http://objstorage-rw-pathslicing:5003
+        url: http://objstorage-ro-pathslicing:5003
 ---
 # Source: swh/templates/storage/configmap.yaml
 apiVersion: v1
 kind: ConfigMap
 metadata:
   namespace: swh-cassandra-next-version
   name: storage-ro-cassandra-configuration-logging
 data:
   logging-gunicorn.json: |
     {
@@ -4604,20 +4687,35 @@
         - kafka-cluster-kafka-brokers:9092
       auto_offset_reset: latest
       group_id: staging-next-version-archive-webhooks
       object_types:
       - origin_visit_status
 ---
 # Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
+  name: objstorage-pathslicing-pvc
+  namespace: swh-cassandra-next-version
+spec:
+  accessModes:
+  - ReadWriteMany
+  resources:
+    requests:
+      storage: 30Gi
+  storageClassName: cephfs
+  volumeMode: Filesystem
+---
+# Source: swh/templates/volumes/persistent-volume-claims.yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
   name: swh-graph-grpc-dataset-example-pvc
   namespace: swh-cassandra-next-version
   labels:
     app: graph-grpc-example
 spec:
   accessModes:
   - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi
@@ -4985,20 +5083,37 @@
       port: 11211
       targetPort: 11211
     - name: metrics
       port: 9150
       targetPort: 9150
 ---
 # Source: swh/templates/objstorage/service.yaml
 apiVersion: v1
 kind: Service
 metadata:
+  name: objstorage-ro-pathslicing
+  namespace: swh-cassandra-next-version
+  labels:
+    app: objstorage-ro-pathslicing
+spec:
+  type: ClusterIP
+  selector:
+    app: objstorage-ro-pathslicing
+  ports:
+    - port: 5003
+      targetPort: 5003
+      name: rpc
+---
+# Source: swh/templates/objstorage/service.yaml
+apiVersion: v1
+kind: Service
+metadata:
   name: objstorage-rw-pathslicing
   namespace: swh-cassandra-next-version
   labels:
     app: objstorage-rw-pathslicing
 spec:
   type: ClusterIP
   selector:
     app: objstorage-rw-pathslicing
   ports:
     - port: 5003
@@ -8480,38 +8595,189 @@
                 path: /metrics
                 port: 9150
             initialDelaySeconds: 5
             periodSeconds: 10
 ---
 # Source: swh/templates/objstorage/deployment.yaml
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   namespace: swh-cassandra-next-version
+  name: objstorage-ro-pathslicing
+  labels:
+    app: objstorage-ro-pathslicing
+spec:
+  revisionHistoryLimit: 2
+  selector:
+    matchLabels:
+      app: objstorage-ro-pathslicing
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxSurge: 1
+  template:
+    metadata:
+      labels:
+        app: objstorage-ro-pathslicing
+      annotations:
+        checksum/config: 71f02bfaba97500abb88050f40502c4ad09338eae7585ab66c7202fa2838bbe5
+        checksum/config-logging: cd8bff14f691fefde95391e737e2658637337c3b0033bd479f315a822701b316
+        checksum/config-utils: 94d255131467f84bef964a4c72b2b792c5ebaf711bb1c77829d7cd1007a8ac22
+    spec:
+      securityContext:
+        fsGroup: 1005
+      nodeSelector:
+        kubernetes.io/hostname: rancher-node-staging-rke2-metal01
+      affinity:
+        nodeAffinity:
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+            - matchExpressions:
+              - key: swh/objstorage
+                operator: In
+                values:
+                - "true"
+      priorityClassName: swh-cassandra-next-version-frontend-rpc
+      initContainers:
+        - name: prepare-configuration
+          image: container-registry.softwareheritage.org/swh/infra/swh-apps/utils:20250211.1
+          imagePullPolicy: IfNotPresent
+          command:
+          - /entrypoints/prepare-configuration.sh
+          env:
+
+          volumeMounts:
+          - name: configuration
+            mountPath: /etc/swh
+          - name: configuration-template
+            mountPath: /etc/swh/configuration-template
+          - name: config-utils
+            mountPath: /entrypoints
+            readOnly: true
+      containers:
+        - name: objstorage-ro-pathslicing
+          resources:
+            requests:
+              memory: 512Mi
+              cpu: 500m
+          image: container-registry.softwareheritage.org/swh/infra/swh-apps/objstorage:20250211.3
+          imagePullPolicy: IfNotPresent
+          ports:
+            - containerPort: 5003
+              name: rpc
+          readinessProbe:
+            httpGet:
+              path: /
+              port: rpc
+            initialDelaySeconds: 15
+            failureThreshold: 30
+            periodSeconds: 5
+          livenessProbe:
+            tcpSocket:
+              port: rpc
+            initialDelaySeconds: 10
+            periodSeconds: 5
+          command:
+          - /bin/bash
+          args:
+          - -c
+          - /opt/swh/entrypoint.sh
+          env:
+            - name: WORKERS
+              value: "2"
+            - name: THREADS
+              value: "2"
+            - name: TIMEOUT
+              value: "60"
+            - name: STATSD_HOST
+              value: prometheus-statsd-exporter
+            - name: STATSD_PORT
+              value: "9125"
+            - name: STATSD_TAGS
+              value: deployment:objstorage-ro-pathslicing
+            - name: STATSD_SERVICE_TYPE
+              value: objstorage-ro-pathslicing
+            - name: SWH_CONFIG_FILENAME
+              value: /etc/swh/config.yml
+            - name: SWH_LOG_LEVEL
+              value: "INFO"
+            - name: SWH_LOG_CONFIG_JSON
+              value: /etc/swh/logging/logging-gunicorn.json
+            - name: SWH_SENTRY_ENVIRONMENT
+              value: staging
+            - name: SWH_MAIN_PACKAGE
+              value: swh.objstorage
+            - name: SWH_SENTRY_DSN
+              valueFrom:
+                secretKeyRef:
+                  name: common-secrets
+                  key: objstorage-sentry-dsn
+                  # 'name' secret should exist & include key
+                  # if the setting doesn't exist, sentry pushes will be disabled
+                  optional: true
+            - name: SWH_SENTRY_DISABLE_LOGGING_EVENTS
+              value: "true"
+          volumeMounts:
+          - name: configuration
+            mountPath: /etc/swh
+          - name: configuration-logging
+            mountPath: /etc/swh/logging
+          - name: pathslicing-rw
+            mountPath: /srv/swh/objects
+            readOnly: true
+
+      volumes:
+      - name: configuration
+        emptyDir: {}
+      - name: configuration-template
+        configMap:
+          name: objstorage-ro-pathslicing-configuration-template
+          items:
+          - key: "config.yml.template"
+            path: "config.yml.template"
+      - name: configuration-logging
+        configMap:
+          name: objstorage-ro-pathslicing-configuration-logging
+          items:
+          - key: "logging-gunicorn.json"
+            path: "logging-gunicorn.json"
+      - name: config-utils
+        configMap:
+          name: config-utils
+          defaultMode: 0555
+      - name: pathslicing-rw
+        persistentVolumeClaim:
+          claimName: objstorage-pathslicing-pvc
+---
+# Source: swh/templates/objstorage/deployment.yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  namespace: swh-cassandra-next-version
   name: objstorage-rw-pathslicing
   labels:
     app: objstorage-rw-pathslicing
 spec:
   revisionHistoryLimit: 2
   selector:
     matchLabels:
       app: objstorage-rw-pathslicing
   strategy:
     type: RollingUpdate
     rollingUpdate:
       maxSurge: 1
   template:
     metadata:
       labels:
         app: objstorage-rw-pathslicing
       annotations:
-        checksum/config: 91292ffc846f0c59408d566d3f460483d5cb29c3bda9ee7d121b3be04dc67c68
+        checksum/config: 90e30ca3c2599e3d9dd0814ee18199f3a8327cbc7db090d29bb13a3cb77cf777
         checksum/config-logging: 133d6dfe9a941649b0b690857989ce74b7577bc3b58f9bc1e17fcc1784ae85fa
         checksum/config-utils: 94d255131467f84bef964a4c72b2b792c5ebaf711bb1c77829d7cd1007a8ac22
     spec:
       securityContext:
         fsGroup: 1005
       nodeSelector:
         kubernetes.io/hostname: rancher-node-staging-rke2-metal01
       affinity:
         nodeAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
@@ -8623,32 +8889,22 @@
         configMap:
           name: objstorage-rw-pathslicing-configuration-logging
           items:
           - key: "logging-gunicorn.json"
             path: "logging-gunicorn.json"
       - name: config-utils
         configMap:
           name: config-utils
           defaultMode: 0555
       - name: pathslicing-rw
-        ephemeral:
-          volumeClaimTemplate:
-            metadata:
-              labels:
-                type: persistent-volume
-            spec:
-              accessModes:
-              - ReadWriteOnce
-              resources:
-                requests:
-                  storage: 10Gi
-              storageClassName: local-persistent
+        persistentVolumeClaim:
+          claimName: objstorage-pathslicing-pvc
 ---
 # Source: swh/templates/provenance/deployment.yaml
 apiVersion: apps/v1
 kind: Deployment
 metadata:
   namespace: swh-cassandra-next-version
   name: provenance-graph-granet
   labels:
     app: provenance-graph-granet
 spec:
@@ -9951,21 +10207,21 @@
       app: storage-ro-cassandra
   strategy:
     type: RollingUpdate
     rollingUpdate:
       maxSurge: 1
   template:
     metadata:
       labels:
         app: storage-ro-cassandra
       annotations:
-        checksum/config: 5da7dff03c99fc7b8b7c7bba9bcca019f750d07da00ae2ac13df5845229529a3
+        checksum/config: 82ddd066ef3b39ba30a9377b2e3d5f34e9c6771d244cb9966170d8821659043e
         checksum/config-logging: 539b1f63c51f751ac609a212d1d29b53ccee1fe8861d4b40cf156cabdbbcc9af
         checksum/backend-utils: 32b7a7114be63a5fc6c3745bbc5fc4a9722e45283bafbb74fd640ffc9fe052a2
         checksum/config-utils: 94d255131467f84bef964a4c72b2b792c5ebaf711bb1c77829d7cd1007a8ac22
     spec:
       affinity:
         nodeAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
             nodeSelectorTerms:
             - matchExpressions:
               - key: swh/storage


------------- diff for environment production namespace swh -------------

No differences


------------- diff for environment production namespace swh-cassandra -------------

No differences

[4] testing read/write from objstorage instances

swhstorage@objstorage-read-write-7489cfd58-ddmgd:/opt/swh$ ls /srv/swh/objects/
test
swhstorage@objstorage-read-write-7489cfd58-ddmgd:/opt/swh$ ls -lah /srv/swh/objects/
total 0
drwxrwxrwx 1 root       root        8 Mar  7 12:59 .
drwxr-xr-x 1 root       root       14 Mar  7 12:56 ..
-rw-r--r-- 1 swhstorage swhstorage  0 Mar  7 12:59 test

swhstorage@objstorage-read-only-564f4bd647-899rq:/opt/swh$ ls -lah /srv/swh/objects/
total 0
drwxrwxrwx 1 root       root        8 Mar  7 12:59 .
drwxr-xr-x 1 root       root       14 Mar  7 12:57 ..
-rw-r--r-- 1 swhstorage swhstorage  0 Mar  7 12:59 test

[5] helm diff on pvc objstorage deployment

tony@kessel:~/work/swh/sysadm-environment/swh-charts (mr/centralize-pv-pvcs) $ make cc-helm-diff | colordiff
Switched to branch 'production'
Switched to branch 'mr/centralize-pv-pvcs'
[cluster-components] Comparing changes between branches production and mr/centralize-pv-pvcs...
Your branch is ahead of 'origin/production' by 10 commits.
  (use "git push" to publish your local commits)
[cluster-components] Generate config in production branch for cluster-components/values/admin-rke2.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/archive-production-rke2.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/archive-staging-rke2.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/default.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/gitlab-production.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/gitlab-staging.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/local-cluster.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/rancher.yaml...
[cluster-components] Generate config in production branch for cluster-components/values/test-staging-rke2.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/admin-rke2.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/archive-production-rke2.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/archive-staging-rke2.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/default.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/gitlab-production.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/gitlab-staging.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/local-cluster.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/rancher.yaml...
[cluster-components] Generate config in mr/centralize-pv-pvcs branch for cluster-components/values/test-staging-rke2.yaml...


------------- diff for cluster-components/values/admin-rke2.yaml -------------

No differences


------------- diff for cluster-components/values/archive-production-rke2.yaml -------------

No differences


------------- diff for cluster-components/values/archive-staging-rke2.yaml -------------

--- /tmp/swh-chart.cluster-components.zDrXEMJW/archive-staging-rke2.yaml.before 2025-03-07 16:57:41.174804727 +0100
+++ /tmp/swh-chart.cluster-components.zDrXEMJW/archive-staging-rke2.yaml.after  2025-03-07 16:57:41.722787196 +0100
@@ -142,20 +142,35 @@
   name: redis-svix-next-version-ext-config
   namespace: swh-cassandra-next-version
   labels:
     app: redis-svix-next-version
 data:
   redis-additional.conf: |
     save 60 500
     appendonly yes
     appendfsync everysec
 ---
+# Source: cluster-config/templates/volumes/persistent-volume-claims.yaml
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: objstorage-pathslicing-pvc
+  namespace: swh-cassandra-next-version
+spec:
+  accessModes:
+  - ReadWriteMany
+  resources:
+    requests:
+      storage: 10Gi
+  storageClassName: local-persistent
+  volumeMode: Filesystem
+---
 # Source: cluster-config/templates/scrape-cluster-metrics/service.yaml
 apiVersion: v1
 kind: Service
 metadata:
   namespace: swh-cassandra-next-version
   name: kafka-next-version-internal-servers-service
   labels:
     app.kubernetes.io/instance: kafka-cluster
     app.kubernetes.io/name: kafka-exporter
 spec:


------------- diff for cluster-components/values/default.yaml -------------

No differences


------------- diff for cluster-components/values/gitlab-production.yaml -------------

No differences


------------- diff for cluster-components/values/gitlab-staging.yaml -------------

No differences


------------- diff for cluster-components/values/local-cluster.yaml -------------

No differences


------------- diff for cluster-components/values/rancher.yaml -------------

No differences


------------- diff for cluster-components/values/test-staging-rke2.yaml -------------

No differences

tony@kessel:~/work/swh/sysadm-environment/swh-charts (mr/centralize-pv-pvcs) $ make swh-helm-diff | colordiff
Switched to branch 'production'
Switched to branch 'mr/centralize-pv-pvcs'
Switched to branch 'production'
Switched to branch 'mr/centralize-pv-pvcs'
[swh] Comparing changes between branches production and mr/centralize-pv-pvcs (per environment)...
Your branch is ahead of 'origin/production' by 10 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment staging...
Your branch is ahead of 'origin/production' by 10 commits.
  (use "git push" to publish your local commits)
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace next-version...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...
[swh] Generate config in mr/centralize-pv-pvcs branch for environment production...


------------- diff for environment staging namespace swh -------------

No differences


------------- diff for environment staging namespace swh-cassandra -------------

No differences


------------- diff for environment staging namespace next-version -------------

--- /tmp/swh-chart.swh.kqsfjQZ0/staging-next-version.before     2025-03-07 17:13:11.285049455 +0100
+++ /tmp/swh-chart.swh.kqsfjQZ0/staging-next-version.after      2025-03-07 17:13:11.949028214 +0100
@@ -4687,35 +4687,20 @@
         - kafka-cluster-kafka-brokers:9092
       auto_offset_reset: latest
       group_id: staging-next-version-archive-webhooks
       object_types:
       - origin_visit_status
 ---
 # Source: swh/templates/volumes/persistent-volume-claims.yaml
 apiVersion: v1
 kind: PersistentVolumeClaim
 metadata:
-  name: objstorage-pathslicing-pvc
-  namespace: swh-cassandra-next-version
-spec:
-  accessModes:
-  - ReadWriteMany
-  resources:
-    requests:
-      storage: 10Gi
-  storageClassName: local-persistent
-  volumeMode: Filesystem
----
-# Source: swh/templates/volumes/persistent-volume-claims.yaml
-apiVersion: v1
-kind: PersistentVolumeClaim
-metadata:
   name: swh-graph-grpc-dataset-example-pvc
   namespace: swh-cassandra-next-version
   labels:
     app: graph-grpc-example
 spec:
   accessModes:
   - ReadWriteOnce
   resources:
     requests:
       storage: 1Gi


------------- diff for environment production namespace swh -------------

No differences


------------- diff for environment production namespace swh-cassandra -------------

No differences

Refs. swh/infra/sysadm-environment#5593 (closed)

cluster-components/templates/volumes/persistent-volume-claims.yaml

+{{- range $pvcName, $pvcConfig := .Values.volumes.persistentVolumeClaims }}
+{{- if or (not (hasKey $pvcConfig "enabled")) (get $pvcConfig "enabled") -}}
+{{- $namespace := $pvcConfig.namespace | default $.Values.namespace -}}
+{{- $appName := $pvcConfig.appName -}}
+---
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: {{ $pvcName }}
+  namespace: {{ $namespace }}
+  {{- if $appName }}
+  labels:
+    app: {{ $appName }}
+  {{- end }}
+spec:
+  {{- toYaml $pvcConfig.spec | nindent 2 }}
+{{ end }}
+{{ end }}