Skip to content
Snippets Groups Projects

Refactor azure and libcloud based backends

Open Refactor azure and libcloud based backends
douardda/swh-objstorage:refactor-cloud-backends into master
Open David Douard requested to merge douardda/swh-objstorage:refactor-cloud-backends into master
  1. Mar 22, 2023
    • David Douard's avatar
      Refactor the azure backend · f996ee90
      David Douard authored 
      - Drop the AzureCloudObjStorage in favor of using the
  PrefixedAzureObjStorage (with "" prefix)
- Drop support for (deprecated) configuration based on account name and key
- Adapt a bit the Azurite based tests and add a couple of test cases
  (especially non-sha1 primary hash and a 32 bits prefix test case).
      f996ee90
    • David Douard's avatar
      Refactor CloudObjStorage provider logic · c54f0751
      David Douard authored 
      Simplify a bit the cloud provider logic behinf CloudObjStorage:

- get rid of the _get_provider() method in favor of a simple constructor
  "provider" argument,
- deprecate the 2 specialized classes AwsCloudObjStorage and
  OpenStackCloudObjStorage; one should directly use the CloudObjStorage
  instead,
- deprecate "s3" and "swift" objstorage factory cls,
- adapt tests accordingly.
      c54f0751
    • David Douard's avatar
      Allow to name objstorage instances · 964d4721
      David Douard authored 
      this will allow to use this name to add a statsd probe counting which backend
has been successful in a multiplexed backend.
      964d4721
    • David Douard's avatar
      Make the primary hash of an objstorage configurable · 3c08aac2
      David Douard authored 
      The primary hash (aka the has used as primary key for the objstorage
backend) used to be hardcoded as sha1, this makes it configurable among
supported hashes (sha1, sha1_git, sha256 and blake2s256).

For the remote api to work, a new `/config` endpoint has been added
allowing a RPC client to ask the server for a config (generic dict), in
which the "primary_hash" key is used to configure the client with the same
primary hash as the server. If not such endpoint is available, defaults
to "sha1".
      3c08aac2