Find a way to properly open the kafka brokers to the internet
The kafka brokers need to be accessible from the internet, so our mirrors can subscribe to the topics and process messages.
We need to figure out:
- frontend/proxying
- TLS
- authentication
- authorization
For reference:
- proxy-based deployment with TLS offloading : https://medium.com/code-tech/kafka-in-aws-with-ssl-offloading-using-load-balancer-c337da1435c3
- Authentication: https://docs.confluent.io/current/kafka/authentication_sasl/index.html
- Authorization: https://docs.confluent.io/current/kafka/authorization.html
There's a strong chance that the journal code will need to be adapted to allow passing the proper settings to the kafka libraries.
Migrated from T1829 (view on Phabricator)