Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Register
  • Sign in
  • S swh-loader-git
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 28
    • Issues 28
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 4
    • Merge requests 4
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Platform
  • Development
  • swh-loader-git
  • Issues
  • #3654
Closed
Open
Issue created Oct 14, 2021 by Nicolas Dandrimont@olasdMaintainer

loader git: load revisions in topological order

Right now, the git loader loads revision objects in the order that they come in the packfile sent by upstream. This means that if the loader is interrupted, for whatever reason, in the middle of adding revisions, we will have loaded a set of revisions with no guarantees that their parents have been properly added.

This means that we cannot use a simple global lookup of revision objects to reduce the effort of loading "undeclared" forks: we cannot consider that any revision currently present in the SWH archive is complete.

If we can ensure that existing revision objects have been properly loaded, including their parents, making the git loader sort revision objects in topological order before adding them would allow us to use a global lookup of revision objects to reduce the size of packfiles received from the server (instead of restricting ourselves to earlier snapshots of the same origin).


Migrated from T3654 (view on Phabricator)

Assignee
Assign to
Time tracking