Add a new extra_headers attribute on Revision and use it for computing
the revision's id instead of extract it from the metadata field.

Only accept (bytes, bytes) as extra_header.

Add a post init hook to Revision to initialize this new attribute from
given metadata, if any, for bw compat.

Also amend the revision_d hyptothesis strategy to generate extra_headers.

When Software Heritage persistent identifiers were introduced, they were not
yet abbreviated as SWHIDs.

Now that abbreviation is growing adoption, rename some functions and types in
swh.model.identifiers for consistency:

  - PersistentId -> SWHID

  - persistent_identifier -> swhid

  - parse_persistent_identifier -> parse_swhid

Backward compatibility with previous naming is maintained but deprecation
warnings are introduced to encourage the use of the new names.

Numerous variables in swh.model codebase have also been renamed accordingly.

Also rework and improve documentation.

Related to T2310

this aims at preventing constant usage of isinstance() based dispatch
code when writing generic code handling model entities.

For example, the "object_type" argument of JournalWriter.write_addition() has
become superflous now we only pass model entities, etc.

This idea comes olasd's reading of mypy doc:

  https://mypy.readthedocs.io/en/latest/literal_types.html#tagged-unions

This comes with a refactoring of from_dict.DiskBackedContent to make
it *not* inherit from model.Content: object_type being Final, it cannot
be overloaded.

Related to T2310

Related to T2310

This is not used. This is broken storage wise (origin-visit-add does not deal
correctly with it and it so happens there is no test around it).

And finally, this will soon go away with T2310.

And adapt the revisions_d() strategy accordingly.

This can be None, according to the model.

for the sake of consistency.

thus in TimestampWithTimezone.from_dict(). This is needed to help consuming
existing (invalid) messages from kafka.

Warning: tests added in this revision do not cover the whole
normalize_timestamp() function.

This is a step forward having model objects, declared as frozen, immutable.

This requires attrs_strict >= 0.0.7.

the visit attribute is expected to be strictly positive.

this attribute is not an intrinsic property of a content object, so it
should not be used when comparing or hashing.

Simply add a BaseModel.anonymize() method. Default implementation returns
None, meaning the object is not anonymizable.

For Person, the method returns a Person whith hashed fullname (and unset
name and email).

For Revision and Release, the method returns an anonymized version of
the object, i.e. with instance of Person replaced by anonymized ones.

Stefano Zacchiroli authored 4 years ago and Roberto Di Cosmo committed 4 years ago

Reviewers: rdicosmo

Reviewed By: rdicosmo

Differential Revision: https://forge.softwareheritage.org/D3108

Production should only use aware datetimes.

Closes T1003

 - merge grammars into a single one
 - explain better that SWHIDs are made up of core identifier + qualifiers
 - separate qualifier into context and fragment onex
 - add reference to swh-identify

Closes T2379

Make it possible to generate Content and SkippedContent under different
object types (namely "content" and "skipped_content").

Default to False to keep backward compat.

so one can choose not to generate some of the object types.

Blacklist "origin_visit_status" by default to prevent breaking dependent
packages' tests.

Related to T2367

and user required=True to check that, as it is the preferred way

hat tip to @rdicosmo for noticing