Skip to content

Add support for slices when getting objects from the objstorage.

Currently get methods only supports returning the full blob. We should add new parameters (eg. start and end) to specify the range of bytes the caller wants.

These get methods are defined in swh-objstorage/swh/objstorage/objstorage.py (the abstract base class) and in three different backends:

  • swh/objstorage/objstorage_pathslicing.py manipulates a file object, so it's a matter of using seek() and read().
  • swh/objstorage/objstorage_in_memory.py manipulates Python bytes objects, so it only needs slicing.
  • swh/objstorage/objstorage_rados.py uses RADOS, so it's a bit more tedious. Fortunately, it already uses the slicing logic of RADOS (self.ioctx.read(_obj_id, offset, READ_SIZE)), so it's a matter of changing values of the arguments to self.ioctx.read.

Migrated from T1447 (view on Phabricator)