Skip to content
Snippets Groups Projects
  1. Feb 02, 2023
  2. Jan 20, 2023
  3. Jan 18, 2023
  4. Jan 17, 2023
  5. Dec 19, 2022
  6. Dec 08, 2022
  7. Dec 07, 2022
    • Antoine Lambert's avatar
      replay: Copy dir states and external paths in copy_from operations · b016f654
      Antoine Lambert authored
      Copied directories might have externals so we also need to copy states
      and update external paths in case externals list is later modified.
      b016f654
    • Antoine Lambert's avatar
      svn: Use urllib.parse.quote to percent encode svn URLs · fc78f574
      Antoine Lambert authored
      In order to detect all ascii characters that must be percent encoded
      in svn URLs, add a brute force test and use urllib.parse.quote in
      quote_svn_url function.
      fc78f574
    • Antoine Lambert's avatar
      utils: Raise ValueError when external definition could not be parsed · f377e9f7
      Antoine Lambert authored
      Such case can happen when an external definition is malformed.
      
      Previously, the parsed malformed external was added to the directory state
      with an empty external URL which could lead to unexpected side effects like
      removing all previously exported valid externals.
      f377e9f7
    • Antoine Lambert's avatar
      replay: Simplify FileEditor implementation · 301b31e9
      Antoine Lambert authored
      Instead of maintaining file state based on svn properties across revisions
      replay and trying to reconstruct the same file as with a svn export operation
      after applying text deltas, prefer to simply export the file from the currently
      processed revision when closing the associated file editor.
      
      This greatly simplify the replay module implementation while approximatively
      keeping the same performance as before.
      
      Also add a test that would fail without these changes.
      
      Related to T4673
      301b31e9
  8. Dec 05, 2022
  9. Nov 25, 2022
    • Antoine Lambert's avatar
      replay: Add more debug logs · e35f800a
      Antoine Lambert authored
      Add more debug logs to the replay module to ease detection of issues.
      Nevertheless, as those are quite verbose, only display them when setting
      debug parameter of the loader to True.
      e35f800a
  10. Nov 23, 2022
  11. Nov 22, 2022
    • Antoine Lambert's avatar
      loader: Add logs displaying path differences after revision replay · a843858b
      Antoine Lambert authored
      When a tree computation divergence is detected after replaying a revision
      add debug logs displaying the paths that differ or are missing between the
      reconstructed repository filesystem and the exported one at that specific
      revision.
      
      It should help to gain some time when debugging such issues.
      a843858b
  12. Oct 31, 2022
    • Antoine Lambert's avatar
      replay: Ensure copyfrom operations are properly handled · 04566a7f
      Antoine Lambert authored
      A subversion revision can contain new directories and files copied from
      ancestor revisions but those were not perfectly handled in the commit
      editor used to reconstruct the repository filesystem when replaying
      revisions.
      
      In particular previous implementation could not handle the case where a
      path copied from an ancestor revision is replaced in a same commit (for
      instance replacing a directory by a file with the same name).
      
      These changes ensure that info about source path and source revision from
      which a path is copied is passed to the commit editor methods as paramaters
      in order to let them handle the copies but also that the replace operations
      will be correctly replayed.
      
      It also prevents OS error "Too many open files" when a really large files
      tree is copied from an ancestor revision.
    • Antoine Lambert's avatar
      loader: Compress dump file and rework truncated dump handling · d24ba1a5
      Antoine Lambert authored
      When dumping a subversion repository to file before loading it, compress
      that file using gzip while producing it. It enables to save significant
      disk space while dumping a large repository.
      
      Also rework the way how truncated dump is handled now dump file is
      compressed by providing the expected max revision number to be loaded
      by svnadmin. If the number of loaded revisions matches, we can safely
      continue the partial loading of the repository.
      d24ba1a5
  13. Oct 28, 2022
  14. Oct 25, 2022
  15. Oct 19, 2022
  16. Oct 18, 2022
  17. Oct 17, 2022
    • Antoine Lambert's avatar
      replay: Use swh.model.from_disk.Directory.collect · aaa82617
      Antoine Lambert authored
      Instead of maintaining a set of modified paths for each replayed revision,
      use the swh.model.from_disk.Directory.collect method which performs the same
      task by returning added or modified contents and directories since the last
      collect operation.
      aaa82617
  18. Sep 30, 2022
  19. Jun 17, 2022
  20. Jun 01, 2022
  21. May 20, 2022
  22. May 09, 2022
  23. May 05, 2022
  24. May 03, 2022
  25. Apr 27, 2022
  26. Apr 26, 2022
    • Antoine Lambert's avatar
      tasks: Fix and simplify implementation · b4bca94f
      Antoine Lambert authored
      Recent changes in swh-scheduler add new parameters to the celery tasks
      produced from swh.scheduler.model.ListedOrigin instances.
      
      Those new parameters were not properly handled in the celery tasks
      implementation for svn loading but also in some svn loader ones.
      
      So ensure to handle any new parameters by not hardcoding the expected
      ones in task signatures but also allowing any extra keyword parameters
      in loader constructors.
      
      Also remove explicit setting of visit_date if none has been provided
      as task parameter as it is already handled in loader constructor.
      
      Related to T4187
      b4bca94f
    • vlorentz's avatar
      Bump mypy to v0.942 · 91dc7ad9
      vlorentz authored
      91dc7ad9
Loading