Skip to content

Make the kafka group_id prefix configurable in the config file

rather than hardcoding it to 'swh-dataset-export-', use the 'group_id' value from the 'journal' section of the config file as prefix, if given 9otherwise default to the former value).

This is needed because current auth policy of swh kafka cluster only allow group_id to start with the actual login for authenticated connection. So we need to be able to specify this group_id prefix.

Depends on !35 (closed).


Migrated from D7382 (view on Phabricator)

Merge request reports

Loading