Skip to content
Snippets Groups Projects
  1. Jan 12, 2021
    • vlorentz's avatar
      test_identifiers: Reorder SWHID tests. · 1d0c3212
      vlorentz authored
      They were mixed in with snapshot tests.
      1d0c3212
    • vlorentz's avatar
      test_identifiers: Make sure that... · 731d10d3
      vlorentz authored
      test_identifiers: Make sure that {directory,revision,release,snapshot}_identifier() doesn't just return a value from the dict.
      
      For example, before this commit, you could replace the code of
      revision_identifier() with this:
      
      def release_identifier(release):
          return release.get("id", b"")
      
      and all tests would still pass.
      731d10d3
  2. Jan 04, 2021
  3. Dec 30, 2020
    • Stefano Zacchiroli's avatar
      SWHID parsing: simplify and deduplicate validation logic · 57468505
      Stefano Zacchiroli authored
      Before this change there was a lot of overlap between parse_swhid() and the
      attrs-based validators in the SWHID class. Also, the validation implementation
      in parse_swhid() was done by hand.
      
      With this change the coarse-grained validation done by parse_swhid() is now
      delegated to a regex. The semantic validation of SWHIDs is left to attrs
      validators. The regex is also exposed as a module attribute, to be used by
      client code that want to syntactically validate SWHIDs without necessarily
      instantiate SWHID classes (we have several other modules doing that already,
      and they are using slightly different hand-made regexs, which isn't great).
      
      As part of this change we also clean up the use of ValidationError exceptions,
      systematically passing the problematic parts of SWHID as arguments, and uniform
      error messages.
      
      This change also brings some speed up in SWHID parsing. On a benchmark parsing
      ~30 M valid SWHIDs, the previous implementation took ~3:06 minutes, the new one
      ~2:50 minutes, or a ~9% speedup.
      
      Closes T2788
      57468505
  4. Dec 15, 2020
    • vlorentz's avatar
      model: Make all classes slotted. · 76b744e0
      vlorentz authored
      Unfortunately, sphinx (actually, autodoc) only picks up attributes if
      they fall in any of these cases:
      
      1. are enum variants
      2. are in slots
      3. are in __dict__
      4. have an annotation
      5. are found using its custom parser
      
      (see get_object_members in sphinx/ext/autodoc/importer.py)
      
      In theory, option 5 should work for us; unfortunately, autodoc only
      asks the parser the list of members with a comment.
      And it's not easy to adapt it to ask the parser for all members,
      because said parser (sphinx/pycode/parser.py) does not return the class
      qualname (aka. namespace) for members without comments.
      
      So, as I don't want to change the interface of sphinx.pycode.parser,
      this commit switches to relying on option 3, by adding __slots__ for
      all attr classes.
      
      Additionally, this might have some performance/memory improvement
      (though I did not check) and will further avoid mutation of these
      objects.
      76b744e0
  5. Nov 16, 2020
  6. Nov 12, 2020
  7. Nov 10, 2020
  8. Oct 27, 2020
  9. Oct 26, 2020
  10. Oct 23, 2020
  11. Oct 14, 2020
    • Nicolas Dandrimont's avatar
      Make revision/release identifiers explicitly the hash of a manifest · 9224c8ca
      Nicolas Dandrimont authored
      This collapses the shared logic between these two identifier computations into a
      few more explicit steps:
       - generate data for the manifest (in either identifier computation);
       - format the manifest (in the new format_manifest function);
       - hash the manifest (in the new hash_manifest function).
      
      This will enable reusing this logic for more object types, as well as stronger
      typing for the manifest computation.
      9224c8ca
  12. Oct 08, 2020
    • vlorentz's avatar
      Add a 'unique_key' method on model objects · a251df2e
      vlorentz authored
      that returns a value suitable for unicity constraints.
      
      Motivation:
      
      * this is somewhat more of a model concern than a journal/kafka
        concern IMO
      * this is one step toward adding support for non-model objects in
        KafkaJournalWriter
      
      Implementation of the unique_key methods comes from
      `swh.journal.serializers.object_key`.
      v0.7.1
      a251df2e
  13. Oct 05, 2020
  14. Oct 02, 2020
  15. Sep 29, 2020
  16. Sep 25, 2020
  17. Sep 18, 2020
  18. Sep 17, 2020
  19. Sep 10, 2020
  20. Aug 25, 2020
  21. Aug 14, 2020
    • vlorentz's avatar
      model: Raise error on naive datetimes. · 6dd6acec
      vlorentz authored
      We may unknowingly pass naive datetimes to the storage through them,
      causing the underlying DB to assign them a timezone that might not match
      the actual one.
      
      It already happens in swh.model and swh.loader.package tests.
      6dd6acec
  22. Aug 07, 2020
  23. Aug 06, 2020
  24. Aug 04, 2020
  25. Jul 30, 2020
  26. Jul 29, 2020
Loading