Skip to content
Snippets Groups Projects
Forked from Platform / Development / swh-vault
28 commits behind the upstream repository.
Antoine Lambert's avatar
Antoine Lambert authored
Previously when cooking a directory, contents bytes were fetched
sequentially which could take a good amount of time for large
directories.

In order to speedup the cooking process, retrieve the contents bytes
in parallel with the help of the concurrent.futures module from the
Python standard library which fits particularly well for making loops
of I/O-bound tasks concurrent and for issuing tasks asynchronously.
295e4138
History

Software Heritage - Vault

User-facing service that allows to retrieve parts of the archive as self-contained bundles (e.g., individual releases, entire repository snapshots, etc.) The creation of a bundle is called "cooking" a bundle.

Architecture

The vault is made of two main parts:

  1. a stateful RPC server called the backend
  2. Celery tasks, called cookers