Currently, the writing in the azure objstorage adds extra headers to the blobs. Which prevents the correct reading of those blobs when checking the tarballs.
According to the documentation, we can provide some extra headers to happen to avoid that. From experimenting with the repl on problematic blobs, having no fancy headers unstuck the reading of the blobs.
[swh] Comparing changes between branches production and adapt-deposit-config (per environment)...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment staging, namespace swh...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra...
[swh] Generate config in production branch for environment staging, namespace swh-cassandra-next-version...
[swh] Generate config in adapt-deposit-config branch for environment staging...
[swh] Generate config in adapt-deposit-config branch for environment staging...
[swh] Generate config in adapt-deposit-config branch for environment staging...
Your branch is up to date with 'origin/production'.
[swh] Generate config in production branch for environment production, namespace swh...
[swh] Generate config in production branch for environment production, namespace swh-cassandra...
[swh] Generate config in production branch for environment production, namespace swh-cassandra-next-version...
[swh] Generate config in adapt-deposit-config branch for environment production...
[swh] Generate config in adapt-deposit-config branch for environment production...
[swh] Generate config in adapt-deposit-config branch for environment production...
------------- diff for environment staging namespace swh -------------
No differences
------------- diff for environment staging namespace swh-cassandra -------------
--- /tmp/swh-chart.swh.3Du8uVlJ/staging-swh-cassandra.before 2023-11-22 16:41:36.593051823 +0100
+++ /tmp/swh-chart.swh.3Du8uVlJ/staging-swh-cassandra.after 2023-11-22 16:41:37.293051113 +0100
@@ -377,20 +377,22 @@
port: 5432
name: swh-deposit
user: swh-deposit
password: ${POSTGRESQL_PASSWORD}
cache_uri: memcached:11211
extraction_dir: /tmp/swh-deposit/archive/
max_upload_size: 209715200
azure:
container_name: ""deposit-contents""
connection_string: ""DefaultEndpointsProtocol=https;AccountName=swhdepositstoragestaging;AccountKey=${ACCOUNT_KEY};EndpointSuffix=core.windows.net""
+ content_type: "application/octet-stream"
+ content_encoding: ""
---
# Source: swh/templates/graphql/configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: graphql
namespace: swh-cassandra
data:
# TODO: rename to not have a dot in the name to allow testing
config.yml: |
@@ -14816,21 +14818,21 @@
app: deposit
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
template:
metadata:
labels:
app: deposit
annotations:
- checksum/config: 2626afb0b4c4a1bb08264b20e1c1e5513563198cd9cb907d36a58aec7222eb2c
+ checksum/config: 0f352bed8d1bb6a8f0dd74e7d50777aba7ee28d8c01c8731851bae8f8244ada2
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: swh/deposit
operator: In
values:
- "true"
------------- diff for environment staging namespace swh-cassandra-next-version -------------
No differences
------------- diff for environment production namespace swh -------------
No differences
------------- diff for environment production namespace swh-cassandra -------------
No differencesRefs. swh/infra/sysadm-environment#5129 (closed)
Depends on a future version of the deposit with [1]