Setup efficient and consistent swh-storage pagination [Roadmap - Technical debt]
Milestone ID: 97
- Lead: jayesh
- Priority: high
- Effort: ??
Define and implement an efficient structure for pagination in the data sources for swh-storage.
Pagination in the data sources (eg storage) is not very consistent and client friendly. Defining and implementing an efficient structure will be a good improvement. This will also involve re-factoring some clients.
- Design an efficient pagination architecture
- Refactor obj-storage to implement the pagination
- Identify and refactor existing clients that use swh-storage pagination
- New pagination solution in production for swh-storage
- Existing clients updated to use this solution