Skip to content

staging/deposit: Adapt azure configuration to overwrite headers when writing

Antoine R. Dumont requested to merge adapt-deposit-config into staging

Currently, the writing in the azure objstorage adds extra headers to the blobs. Which prevents the correct reading of those blobs when checking the tarballs.

According to the documentation, we can provide some extra headers to happen to avoid that. From experimenting with the repl on problematic blobs, having no fancy headers unstuck the reading of the blobs.

make swh-helm-diff
[swh] Comparing changes between branches production and adapt-deposit-config (per environment)...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra-next-version...
[swh] Generate config in adapt-deposit-config branch for environment staging...
[swh] Generate config in adapt-deposit-config branch for environment staging...
[swh] Generate config in adapt-deposit-config branch for environment staging...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace swh-cassandra-next-version...
[swh] Generate config in adapt-deposit-config branch for environment production...
[swh] Generate config in adapt-deposit-config branch for environment production...
[swh] Generate config in adapt-deposit-config branch for environment production...


------------- diff for environment staging namespace swh -------------

No differences


------------- diff for environment staging namespace swh-cassandra -------------

--- /tmp/swh-chart.swh.3Du8uVlJ/staging-swh-cassandra.before    2023-11-22 16:41:36.593051823 +0100
+++ /tmp/swh-chart.swh.3Du8uVlJ/staging-swh-cassandra.after     2023-11-22 16:41:37.293051113 +0100
@@ -377,20 +377,22 @@
           port: 5432
           name: swh-deposit
           user: swh-deposit
           password: ${POSTGRESQL_PASSWORD}
     cache_uri: memcached:11211
     extraction_dir: /tmp/swh-deposit/archive/
     max_upload_size: 209715200
     azure:
       container_name: ""deposit-contents""
       connection_string: ""DefaultEndpointsProtocol=https;AccountName=swhdepositstoragestaging;AccountKey=${ACCOUNT_KEY};EndpointSuffix=core.windows.net""
+      content_type: "application/octet-stream"
+      content_encoding: ""
 ---
 # Source: swh/templates/graphql/configmap.yaml
 apiVersion: v1
 kind: ConfigMap
 metadata:
   name: graphql
   namespace: swh-cassandra
 data:
   # TODO: rename to not have a dot in the name to allow testing
   config.yml: |
@@ -14816,21 +14818,21 @@
       app: deposit
   strategy:
     type: RollingUpdate
     rollingUpdate:
       maxSurge: 1
   template:
     metadata:
       labels:
         app: deposit
       annotations:
-        checksum/config: 2626afb0b4c4a1bb08264b20e1c1e5513563198cd9cb907d36a58aec7222eb2c
+        checksum/config: 0f352bed8d1bb6a8f0dd74e7d50777aba7ee28d8c01c8731851bae8f8244ada2
     spec:
       affinity:
         nodeAffinity:
           requiredDuringSchedulingIgnoredDuringExecution:
             nodeSelectorTerms:
             - matchExpressions:
               - key: swh/deposit
                 operator: In
                 values:
                 - "true"


------------- diff for environment staging namespace swh-cassandra-next-version -------------

No differences


------------- diff for environment production namespace swh -------------

No differences


------------- diff for environment production namespace swh-cassandra -------------

No differences

Refs. swh/infra/sysadm-environment#5129 (closed)

Depends on a future version of the deposit with [1]

[1] swh/devel/swh-deposit!421 (merged)

Merge request reports

Loading