staging: Deploy deposit rpc service
- Adapt configuration to allow use of specific backend (e.g. with django-storges extension, either azure [1] or s3 [2] ...)
- swh-apps: Adapt images with the proper dependency
- swh/infra/ci-cd/swh-charts!197 (merged): Adapt chart to override default swh.deposit configuration which uses a specific configuration
- swh/devel/snippets@a34da1cd: Write migration script to upload existing deposit client tarballs to the new backend
- staging: Label nodes with swh/deposit [1]
- Migrate existing data from static staging deposit to dynamic staging deposit
- Activate such staging deposit chart in staging: fail the deposit assumes that deposit tarballs are local to itself (needs adaptation)
- (review) swh/devel/swh-deposit!419 (merged): Adapt deposit so it can check remote tarballs
- swh/devel/swh-deposit!419 (comment 156200): Run deposit within the docker-dev with equivalent remote backend (azurite)
- (review) swh/devel/swh-deposit!419 (merged): Adapt deposit configuration so we can use remote backend too
- (pending review) swh/infra/ci-cd/swh-charts!225 (merged): Simplify chart configuration
- swh/infra/ci-cd/swh-charts!224 (merged): Deploy next version
- Checks: Issue with deposit server (when reading tarball files)
- Fix deposit server issue (static)
- swh/infra/ci-cd/swh-charts!227 (merged): elastic infra: Deploy next version
- Checks (icinga) -> failing...
- Troubleshoot issue: wrong headers stored in the blobstorage which prevents the archive reading (it fails with ContentDecodingError)
-
Fix
- swh/devel/swh-deposit!421 (merged): Allow extra azure configuration in settings.production
- Merge and tag
- swh/infra/ci-cd/swh-charts!240 (merged): Deploy new version
- swh/infra/ci-cd/swh-charts!238 (merged): Configure extra azure configuration for the elastic deposit
- Status
- swh/infra/puppet/puppet-swh-site!662 (merged): Migrate reverse proxy to hit the elastic deposit (through puppet, not manually)
- swh/infra/puppet/puppet-swh-site!663 (merged): Migrate puppet collectable icinga checks to static checks
- Decommission deposit.staging services
-
Checks
- Fix icinga checks (throttling) [4]
- icinga (pergamon): Deploy new icinga plugin version
- icinga check ok
optional:
- Adapt docker-dev configuration with deposit with remote backends
bonus:
- swh/devel/swh-deposit!419 (merged): Adapt swh.deposit directly
[1] https://django-storages.readthedocs.io/en/latest/backends/azure.html
[2] https://django-storages.readthedocs.io/en/latest/backends/amazon-S3.html
[3]
$ environment=staging; for i in $(seq 1 6); do for label in deposit; do kubectl --context archive-$environment-rke2 label --overwrite node rancher-node-staging-rke2-worker$i swh/$label=true; done; done
node/rancher-node-staging-rke2-worker1 labeled
node/rancher-node-staging-rke2-worker2 labeled
node/rancher-node-staging-rke2-worker3 labeled
node/rancher-node-staging-rke2-worker4 labeled
node/rancher-node-staging-rke2-worker5 labeled
node/rancher-node-staging-rke2-worker6 labeled
$ environment=production; for i in $(seq 1 3); do for label in deposit; do kubectl --context archive-$environment-rke2 label --overwrite node rancher-node-metal0$i swh/$label=true; done; done
node/rancher-node-metal01 not labeled
node/rancher-node-metal02 not labeled
node/rancher-node-metal03 not labeled
Note: @gsamson ^
[4] swh/devel/swh-icinga-plugins!55 (merged)
Refs. #4780 (closed)