Snippets Groups Projects

Forked from Platform / Development / swh-vault

27 commits behind, 1 commit ahead of the upstream repository.

to_disk: Use a BFS to recursively list a directory instead of a DFS

Antoine Lambert authored 1 year ago

It enables to push new files to download asynchronously while fetching
sub-directories and thus slightly improve the overall cooking performance.

It should also reduce the memory consumption of the cooking process.

ce58e67c

ce58e67c 1 year ago

Name	Last commit	Last update
docs
swh/vault
.copier-answers.yml
.git-blame-ignore-revs
.gitignore
.pre-commit-config.yaml
AUTHORS
CODE_OF_CONDUCT.md
CONTRIBUTORS
LICENSE
Makefile
README.rst
conftest.py
mypy.ini
pyproject.toml
pytest.ini
requirements-swh-graph.txt
requirements-swh.txt
requirements-test.txt
requirements.txt
setup.cfg
tox.ini

Software Heritage - Vault

User-facing service that allows to retrieve parts of the archive as self-contained bundles (e.g., individual releases, entire repository snapshots, etc.) The creation of a bundle is called "cooking" a bundle.

Architecture

The vault is made of two main parts:

a stateful RPC server called the backend
Celery tasks, called cookers