As NamedTuple inherits from tuple, msgpack serializes it like a tuple,
which makes it indistinguishable from a tuple when deserializing,
which is an issue for the RPC API.

Add a new extra_headers attribute on Revision and use it for computing
the revision's id instead of extract it from the metadata field.

Only accept (bytes, bytes) as extra_header.

Add a post init hook to Revision to initialize this new attribute from
given metadata, if any, for bw compat.

Also amend the revision_d hyptothesis strategy to generate extra_headers.

When Software Heritage persistent identifiers were introduced, they were not
yet abbreviated as SWHIDs.

Now that abbreviation is growing adoption, rename some functions and types in
swh.model.identifiers for consistency:

  - PersistentId -> SWHID

  - persistent_identifier -> swhid

  - parse_persistent_identifier -> parse_swhid

Backward compatibility with previous naming is maintained but deprecation
warnings are introduced to encourage the use of the new names.

Numerous variables in swh.model codebase have also been renamed accordingly.

Also rework and improve documentation.

thus in TimestampWithTimezone.from_dict(). This is needed to help consuming
existing (invalid) messages from kafka.

Warning: tests added in this revision do not cover the whole
normalize_timestamp() function.

- blackify all the python files,
- enable black in pre-commit,
- add a black tox environment.

It should be cheap enough to do it, and it makes tests easier.

Daniele Serafini authored 5 years ago and Stefano Zacchiroli committed 5 years ago

... from test_persistent_identifier.

Closes T1986

Related T1152

This commit changes the swh pids representation returned by the
parse_persistent_identifier function from a dict to a named tuple.
Also, the 'object_type' attribute of that named tuple now contains
a full name instead of an abbreviated one.

Closes T1112
Closes T1125

Close T1104
Close D346

Related T1104

As the client of this api might not be aware of what's the right id
key for a given object.

Related T1104

Related T1104

Related T1104

Related T1104

This commits adds support for parsing optional parts in persistent
identifiers. Optional parts are defined the following way:

	<swh_id>[;key1=val1][;key2=val2]

Related T1040

Related T335

Related T335
Related T933

Summary:
Add support for snapshot identifiers

Close T566.
Related to D268.

Test Plan: Unit tests included

Reviewers: zack, #reviewers!

Maniphest Tasks: T566

Differential Revision: https://forge.softwareheritage.org/D277

Related T703

Found by investigating T75

The subversion loader (T680) has shown that throwing floating point values
around for timestamps is a mess waiting to happen.

We now coerce all clients to send us timestamps as integer numbers of seconds
and microseconds, avoiding data losses everywhere.