Better Validation in swh.model
The data model defined in swh.model.model
does the job of declaring the data model, especially expected types for each entity attribute.
However, these types are not currently checked nor validated.
mypy could be used to do more (static) type checking, we are not there yet.
Also, since we are gradually delegating validation of any ingested content (be it from swh.loader
or swh.journal.replayer
) to the model, we need to enforce its strictness:
- enforce the checking/validation capabilities of this model (and static validation using mypy might not be enough here) and
- ensure we have correctly specified, documented and tested API for this model stack (including the
from_dict()
/to_dict()
methods).
One point to decide is whether we want runtime validation or not (or maybe a disengageable one, see attr.set_run_validators() for example).
Some pointers related/useful for this subject:
- https://www.attrs.org/en/stable/api.html#validators (already used, but sparsely),
- projects listed on https://github.com/python-attrs/attrs/wiki/Extensions-to-attrs and especially:
- https://github.com/bloomberg/attrs-strict
Migrated from T2308 (view on Phabricator)
Edited by Phabricator Migration user