Skip to content

wip: git: Group objects per type early to drop the packfile reference asap

Antoine R. Dumont requested to merge generated-differential-D6377-source into generated-differential-D6377-target

Prior to this commit, the loader git would keep the packfile reference and iterate over it multiple times per object type. In this commit, we try to drop the packfile reference earlier to release that reference as soon as possible.

This should reduce the memory pressure on loading very large repository. Assuming the python dict takes less space than the packfile.

Related to #3625 (closed)

Test Plan

tox

(I'm actually checking that it does the right thing on staging with a venv)

it does not. That kills the process almost immediately [1]

  • [1] swh/meta$1177

Migrated from D6377 (view on Phabricator)

Merge request reports