The webapp configuration needs to be adapted to the profile
to not use swh when it's not started

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Using the custom django command

Related to T3280

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Related to T3280

Reverted as the last improvement merge the 2 journal clients

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

This was exposed into 2 services for no good reason.

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Successful deposit upload (from venv):
```
swh deposit upload \
  --url http://deposit.default/1 \
  --username test \
  --password test \
  --archive ../archives/swh-core.tar.gz \
  --metadata ../archives/swh-core.tar.gz.metadata.xml \
  --format json
WARNING:swh.deposit.cli.client:The metadata file provided should contain "<swh:create_origin>" or "<swh:add_to_origin>" tag
{"deposit_id": "2", "deposit_status": "deposited", "deposit_status_detail": null, "deposit_date": "April 17, 2021, 8:28 a.m."}
```

Dedicated deposit loaders output:
```
[deposit-loaders] [2021-04-17 08:28:26,081: INFO/ForkPoolWorker-1] Task swh.deposit.loader.tasks.ChecksDepositTsk[de6faccf-d276-4076-8bf1-5d476d96bb70] succeeded in 0.22092552100366447s: {'status': 'eventful'}
[deposit-loaders] [2021-04-17 08:28:48,517: INFO/ForkPoolWorker-1] Task
swh.loader.package.deposit.tasks.LoadDeposit[c087af8b-c569-4cd1-b32e-f892349c8476]
succeeded in 20.25836444900051s: {'status': 'eventful', 'snapshot_id':
'72d1dee5f4e7871ddb2f6279b2307415b01cbd6c'}
```

Finally report ok (snapshot consistent in deposit swhid with context below):
```
swh deposit status --url http://deposit.default/1 \
  --username test \
  --password test \
  --deposit-id 2 \
  --format json
{"deposit_id": "2", "deposit_status": "done", "deposit_status_detail": "The deposit has been successfully loaded into the Software Heritage archive", "deposit_swh_id": "swh:1:dir:d8e68f38c25c95eb845810cacdba07cc27d2294f", "deposit_swh_id_context": "swh:1:dir:d8e68f38c25c95eb845810cacdba07cc27d2294f;origin=https://softwareheritage.org/6bd7c669-e1bc-4a6f-aa39-cd8061a1740b;visit=swh:1:snp:72d1dee5f4e7871ddb2f6279b2307415b01cbd6c;anchor=swh:1:rev:0959845cfd4b5638348c9147d6905a41e71fdd21;path=/", "deposit_external_id": null}
```

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Up and running:
```
curl http://$(kubectl get services vault -o jsonpath='{.spec.clusterIP}'):5005
SWH Vault API server%
```

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Up and running:
```
$ kubectl get services idx-storage
NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
idx-storage   ClusterIP   10.43.195.217   <none>        5007/TCP   7m3s

$ curl http://$(kubectl get services idx-storage -o jsonpath='{.spec.clusterIP}'):5007
SWH Indexer Storage API server%
```

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

Up and running:
```
$ kubectl get services idx-storage-db
NAME             TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
idx-storage-db   ClusterIP   10.43.160.91   <none>        5432/TCP   7m6s
```

Antoine R. Dumont authored 4 years ago and Vincent Sellier committed 3 years ago

And give some basic usage commands.

- Use buildkit (faster and smarter)
- add parallelism

to test the same kind of configuration as for real environments

It gives some visibility on the images pushed and allow to delete
them