Reschedule pack file too big failing loading task to dedicated queue consumed by large enough workers
Current git loading tasks can fail to be ingested with a [1] 'pack file too big' message. They are then disabled by the scheduler after a number of tryouts (3). Those could be redirected to another queue [2] whose consumption can happen only with large enough workers [3]. (with proper setup like pack file limit set to 32g or something).
- [1]
raise IOError(
f"Pack file too big for repository {origin_url}, "
f"limit is {size_limit} bytes, current size is {cur_size}, "
f"would write {would_write}"
)
-
[3] worker17-18 currently
-
[2] large-repository:swh.loader.git.tasks.UpdateGitRepository
Migrated from T4390 (view on Phabricator)
Edited by Antoine R. Dumont