Skip to content

Add a what is needed for mirrors to watch and act on removal notifications

On removals, we will publish notification on a dedicated Kafka topic. This new notification watcher that should be run on mirrors will listen to the topic and in order:

  1. Create a new masking request in the masking proxy database and add a “decision pending” entry for every SWHID listed in the notification.
  2. Send an email to the mirror operator with commands to either propagate the removal, mask permanently, or unmask all objects.

See the newly added section in usage.rst for a description of the new commands.

Implementation-wise, the passing of information between the mirror notification watcher and the handle-removal-notification command group is done via the masking database. Useful information that were in the removal notification are serialized in YAML to the masking request “reason” field. This will then be retrieved when handling the removal notification, as well as the list of objects that have been masked. While this is a bit of abuse, it feels way simpler that creating a whole new database just for this purpose.

We surely will want to revisit this design once the takedown request processing workflow has been implemented.

Address #20

Edited by Jérémy Bobbio (Lunar)

Merge request reports