Skip to content
Snippets Groups Projects
  1. Mar 03, 2021
  2. Mar 01, 2021
  3. Feb 23, 2021
  4. Feb 19, 2021
    • vlorentz's avatar
      QualifiedSWHID: Replace the 'qualifiers' dict with statically defined attributes · 8e917597
      vlorentz authored
      And store their parsed values (CoreSWHID, tuple of ints, etc.) instead of string.
      8e917597
    • vlorentz's avatar
      Add new class CoreSWHID as an alternative to SWHID/QualifiedSWHID · eba8d84d
      vlorentz authored
      Following the discussion on T3034, we decided to replace SWHID with
      two or three classes:
      
      * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
      * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
      * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
      
      This commit adds the second one
      eba8d84d
    • vlorentz's avatar
      Add new class QualifiedSWHID to replace SWHID, and deprecate the latter. · 690b7f82
      vlorentz authored
      Following the discussion on T3034, we decided to replace SWHID with
      two or three classes:
      
      * QualifiedSWHID to replace the existing SWHID (standard types + qualifiers)
      * CoreSWHID, for "core SWHID" only (standard types + no qualifiers)
      * ExtendedSWHID for internal use in Software Heritage (extra types + no qualifiers)
      
      Since migrating from SWHID will break existing code, this commit uses
      the opportunity to modernize it a little, ie.:
      
      * `keyword`-only constructor, to get rid of the hacky default values for
        `object_type` and `object_id`
      * enum instead of strings for the object type
      * `bytes` instead of an hex string for the object id
      * rename `metadata` to `qualifiers`
      690b7f82
    • vlorentz's avatar
      tests: Clean hashutil._blake2_hash_cache after mocking blake2 functions. · 758eb885
      vlorentz authored
      Depending on the order in which tests are run, these tests may insert
      lambdas with mocked blake2 functions in their closure to be inserted in
      hashutil._blake2_hash_cache; causing all future tests to fail.
      
      While this does not happen with the default order of tests, it does when
      using pytest-xdist.
      758eb885
  5. Jan 26, 2021
  6. Jan 12, 2021
    • vlorentz's avatar
      test_identifiers: Reorder SWHID tests. · 1d0c3212
      vlorentz authored
      They were mixed in with snapshot tests.
      1d0c3212
    • vlorentz's avatar
      test_identifiers: Make sure that... · 731d10d3
      vlorentz authored
      test_identifiers: Make sure that {directory,revision,release,snapshot}_identifier() doesn't just return a value from the dict.
      
      For example, before this commit, you could replace the code of
      revision_identifier() with this:
      
      def release_identifier(release):
          return release.get("id", b"")
      
      and all tests would still pass.
      731d10d3
  7. Dec 30, 2020
    • Stefano Zacchiroli's avatar
      SWHID parsing: simplify and deduplicate validation logic · 57468505
      Stefano Zacchiroli authored
      Before this change there was a lot of overlap between parse_swhid() and the
      attrs-based validators in the SWHID class. Also, the validation implementation
      in parse_swhid() was done by hand.
      
      With this change the coarse-grained validation done by parse_swhid() is now
      delegated to a regex. The semantic validation of SWHIDs is left to attrs
      validators. The regex is also exposed as a module attribute, to be used by
      client code that want to syntactically validate SWHIDs without necessarily
      instantiate SWHID classes (we have several other modules doing that already,
      and they are using slightly different hand-made regexs, which isn't great).
      
      As part of this change we also clean up the use of ValidationError exceptions,
      systematically passing the problematic parts of SWHID as arguments, and uniform
      error messages.
      
      This change also brings some speed up in SWHID parsing. On a benchmark parsing
      ~30 M valid SWHIDs, the previous implementation took ~3:06 minutes, the new one
      ~2:50 minutes, or a ~9% speedup.
      
      Closes T2788
      57468505
  8. Nov 16, 2020
  9. Nov 12, 2020
  10. Nov 10, 2020
  11. Oct 26, 2020
  12. Oct 23, 2020
  13. Oct 08, 2020
    • vlorentz's avatar
      Add a 'unique_key' method on model objects · a251df2e
      vlorentz authored
      that returns a value suitable for unicity constraints.
      
      Motivation:
      
      * this is somewhat more of a model concern than a journal/kafka
        concern IMO
      * this is one step toward adding support for non-model objects in
        KafkaJournalWriter
      
      Implementation of the unique_key methods comes from
      `swh.journal.serializers.object_key`.
      v0.7.1
      a251df2e
  14. Sep 29, 2020
  15. Sep 17, 2020
  16. Aug 14, 2020
    • vlorentz's avatar
      model: Raise error on naive datetimes. · 6dd6acec
      vlorentz authored
      We may unknowingly pass naive datetimes to the storage through them,
      causing the underlying DB to assign them a timezone that might not match
      the actual one.
      
      It already happens in swh.model and swh.loader.package tests.
      6dd6acec
  17. Aug 07, 2020
  18. Jul 29, 2020
  19. Jul 08, 2020
  20. Jul 07, 2020
  21. Jul 06, 2020
    • David Douard's avatar
      Extract the extra_headers from metadata on the Revision model class · a7d9aca2
      David Douard authored
      Add a new extra_headers attribute on Revision and use it for computing
      the revision's id instead of extract it from the metadata field.
      
      Only accept (bytes, bytes) as extra_header.
      
      Add a post init hook to Revision to initialize this new attribute from
      given metadata, if any, for bw compat.
      
      Also amend the revision_d hyptothesis strategy to generate extra_headers.
      v0.4.0
      a7d9aca2
  22. Jul 03, 2020
    • Antoine Lambert's avatar
      identifiers: Rename some functions and types related to SWHIDs · 1ff05161
      Antoine Lambert authored
      When Software Heritage persistent identifiers were introduced, they were not
      yet abbreviated as SWHIDs.
      
      Now that abbreviation is growing adoption, rename some functions and types in
      swh.model.identifiers for consistency:
      
        - PersistentId -> SWHID
      
        - persistent_identifier -> swhid
      
        - parse_persistent_identifier -> parse_swhid
      
      Backward compatibility with previous naming is maintained but deprecation
      warnings are introduced to encourage the use of the new names.
      
      Numerous variables in swh.model codebase have also been renamed accordingly.
      
      Also rework and improve documentation.
      v0.3.8
      1ff05161
  23. Jul 02, 2020
Loading