Skip to content

Storage: Enable to paginate, filter and count snapshot content

Antoine Lambert requested to merge generated-differential-D487-source into generated-differential-D487-target

First draft in order to enable pagination and filtering when one wants to query the content of a snapshot.

That diff adds new optional parameters to the Storage methods returning the content of a snapshot (snapshot_get, snapshot_get_by_origin_visit, snapshot_get_latest):

  • branches_offset: enable to skip a given amount of branches before returning the snapshot content

  • branches_limit: enable to restrain the amount of branches returned in the snapsphot content

  • branches_targets: enable to filter the target type of branches returned in the snapshot branches

I have also added a new method snapshot_branches_count in order to only return the amount of branches according to their target type.

Plugin these new features in swh-web effectively remove the slowdowns observed when browsing an origin with a large amount of branches like for instance https://github.com/v8/v8

Related swh-web#1207 (closed)

Test Plan

Tests showing how to use these new parameters have been added.

Migrated from D487 (view on Phabricator)

Merge request reports