Skip to content

(Incremental) loading of large git repositories fails with an HTTP timeout

In production, doing an incremental load of some large git repositories fails with an HTTP timeout in git-upload-pack.

This problem is two-fold:

  • the git protocol expects an interactive walk of the history to find common ancestors between client and server
  • dulwich's implementation of this interactive walk is not bidirectional, and I think we end up walking the full history of all branches while we're issuing the GET request.

This is especially noticeable on repositories with lots of branches such as https://github.com/mozilla/gecko-dev/

We can fix this by truncating the depth we use to walk the repository history before fetching the repo

Migrated from T1199 (view on Phabricator)