I will use it to locally test incremental loads of large repositories which are already loaded in production, with read-only access of the production storage.
This may also be used in the future to access storages while migrating/replicating.
This is fully tested when the backend storages are empty (by inheriting the usual tests); and never tested when they are not (for now)
I also tried loading linux.git with this config:
storage:
cls: pipeline
steps:
- cls: buffer
- cls: filter
- cls: overlay
storages:
- cls: memory
- cls: remote
url: http://webapp.internal.staging.swh.network:5002/and it didn't use more than 4GB of RAM! (and my machined almost OOMed without the filter proxy), so it seems to work
Migrated from D7838 (view on Phabricator)