Skip to content

api: implement a new /content endpoint that stream the whole content

David Douard requested to merge generated-differential-D1274-source into generated-differential-D1274-target

this stacks a pile of revisions (one of them should be extracted from this sandwich, but it quite painful a task):

  • refactor the SWHRemoteClient so it does not inherit from ObjStorage nor SWHRemoteAPI and implement the remote api accessor as a _proxy attribute instead of inheriting the class to make class instanciation easier and prevent name collision.

  • Also get rid of the ObjStorage inheritance which did not help and add instanciation (argument passing) complexity. This also solve the 'allow_delete' attribute presence which makes no sense for the remote api component.

  • upgrade the server implementation to aiohttp 3 <== this one

  • objstorage: implement the list_content method for most of the storage backends and add a dedicated test.

  • api: implement the list_content method on RemoteObjStorage and update the server's part accordingly.

  • tests: add a test for the ObjStorage.list_content() including testing for pagination (limit and last_obj_id arguments).

    Note: skip the test_list_content for the StripingObjStorage since we have no way to implement pagination support for the list_content() method in this backend.

  • Make iteration over a ObjStorage sorted to be able to have a generic implementation of pagination for the list_content() method.

  • Add a DEFAULT_LIMIT global value for the limit argument of list_content() and use it.

  • pathslicer: implement a better list_content method that prune unnecessaty directories when walking the root path. Add a dedicated unit test.

Migrated from D1274 (view on Phabricator)

Merge request reports