Skip to content

Add support for a priviledged "channel" of topics for non-anonymized objects

David Douard requested to merge generated-differential-D3160-source into generated-differential-D3160-target

This is another approach for implementing anonymized topics. It replaces (with its upcoming counterpart in swh-storage) !171 (closed) and swh-storage!398 (closed).

The idea is to publish on a "public" prefix anonymized objects and have a priviledged prefix for non-anonymized ones. Currently only used for Release and Revision objects.

This adds a "privileged" argument to KafkaJournalWriter.write_addition(s) methods. If True, the topic used to send the message will be

"{prefix}_privileged.{object_type}"

instead of the default

"{prefix}.{object_type}"

On the client side, if both the privileged and unprivileged topics are visible at subscritopn time, only the privileged topics will be subscribed to for object types existing on both "channels".

Migrated from D3160 (view on Phabricator)

Merge request reports