Skip to content

[WIP] Add a (redis-based) validation error reporting facility

David Douard requested to merge generated-differential-D6554-source into generated-differential-D6554-target

This allows to use a ValidatingPorxyStorage that will filter out invalid objects but report them in a redis server.

Key for an invalid object is:

  • its swhid, if any,
  • a string "<object_type>:" if if has a compute_hash() method,
  • a string like "<object_type>:=,=[,...]" if obj.unique_identifier() is a dict

Related to #3693 (closed)

/!\ this is a WIP, no test, no proper requirement, etc. to discuss the idea It does not even properly filter out invalid objects for now.

Also, with the approach of this diff, the object stored in redis is not the message from kafka, but the result of a double msgpack->dict->BaseModel->dict->msgpack transformation which is not great (with filtering/sanitizing stuff embedded in the pipeline).

Finally, having to implement the computation of the key like that is a bit depressing... (who said "SWHID everywhere!"?)

Migrated from D6554 (view on Phabricator)

Merge request reports