Deploy a small publicly available kafka server (with some content) on a staging (+ the related objstorage)
The idea is that I'd like to be able to document the mirror stack so that it "just works" out of the box so people interested in setting up a mirror can test it.
For this, we need a small kafka server with some data in it (say one or 2 small gitab/gitea ingested instances) that is publicly available. Only anonymized topics should be accessible there.
For the content-replayer to work, it will also need an objectstorage, since the content replayer pulls blobs from one objstorage to another (reading sha1s from the content topic). This might be a bit trickier if we want to make it "read-only" to prevent kiddies from playing with it... [edit] the ReadObjStorageFilter is our friend here[/edit]
It might also be useful to have the kafka accessible with and without authentication (to test that authentication layer), but it's not that important.
**Actions : **
-
diff landed and applied on the server -
VIP 128.93.166.40
configured on the firewall -
NAT Port forward of port 9093 from public ip to internal journal0 declared on the firewall -
DNS declaration of broker0.journal.staging.swh.network
in gandi -
Ask to DSI to apply the kafka
firewall profile to128.93.166.40
-
Configure a user to test the pipeline -
Configure a read-only object storage on webapp.staging
Migrated from T2682 (view on Phabricator)