Skip to content

Adapt production vault server configuration to use graph and objstorage

This adds one extra commit with information about the graph (no longer related to this diff).

Related to T3412

Test Plan

octo-diff, only the production vault rpc server configuration is updated to reflect the changes.

Production vault got its configuration changed:

$ bin/octocatalog-diff --to staging vangogh.euwest.azure
...
diff origin/production/vangogh.euwest.azure.internal.softwareheritage.org current/vangogh.euwest.azure.internal.softwareheritage.org
*******************************************
  File[/etc/softwareheritage/vault/server.yml] =>
   parameters =>
     content =>
      @@ -1,3 +1,95 @@
       ---
      +objstorage:
      +  cls: multiplexer
      +  args:
      +    objstorages:
      +    - cls: filtered
      +      args:
      +        storage_conf:
      +          cls: azure-prefixed
      +          args:
      +            accounts:
      +              '0':
      +                account_name: 0euwestswh
      +                api_secret_key: swh::azure::credentials::0euwestswh
      +                container_name: contents
      +              '1':
      +                account_name: 1euwestswh
      +                api_secret_key: swh::azure::credentials::1euwestswh
      +                container_name: contents
      +              '2':
      +                account_name: 2euwestswh
      +                api_secret_key: swh::azure::credentials::2euwestswh
      +                container_name: contents
      +              '3':
      +                account_name: 3euwestswh
      +                api_secret_key: swh::azure::credentials::3euwestswh
      +                container_name: contents
      +              '4':
      +                account_name: 4euwestswh
      +                api_secret_key: swh::azure::credentials::4euwestswh
      +                container_name: contents
      +              '5':
      +                account_name: 5euwestswh
      +                api_secret_key: swh::azure::credentials::5euwestswh
      +                container_name: contents
      +              '6':
      +                account_name: 6euwestswh
      +                api_secret_key: swh::azure::credentials::6euwestswh
      +                container_name: contents
      +              '7':
      +                account_name: 7euwestswh
      +                api_secret_key: swh::azure::credentials::7euwestswh
      +                container_name: contents
      +              '8':
      +                account_name: 8euwestswh
      +                api_secret_key: swh::azure::credentials::8euwestswh
      +                container_name: contents
      +              '9':
      +                account_name: 9euwestswh
      +                api_secret_key: swh::azure::credentials::9euwestswh
      +                container_name: contents
      +              a:
      +                account_name: aeuwestswh
      +                api_secret_key: swh::azure::credentials::aeuwestswh
      +                container_name: contents
      +              b:
      +                account_name: beuwestswh
      +                api_secret_key: swh::azure::credentials::beuwestswh
      +                container_name: contents
      +              c:
      +                account_name: ceuwestswh
      +                api_secret_key: swh::azure::credentials::ceuwestswh
      +                container_name: contents
      +              d:
      +                account_name: deuwestswh
      +                api_secret_key: swh::azure::credentials::deuwestswh
      +                container_name: contents
      +              e:
      +                account_name: eeuwestswh
      +                api_secret_key: swh::azure::credentials::eeuwestswh
      +                container_name: contents
      +              f:
      +                account_name: feuwestswh
      +                api_secret_key: swh::azure::credentials::feuwestswh
      +                container_name: contents
      +        filters_conf:
      +        - type: readonly
      +    - cls: filtered
      +      args:
      +        storage_conf:
      +          cls: remote
      +          args:
      +            url: http://banco.internal.softwareheritage.org:5003/
      +        filters_conf:
      +        - type: readonly
      +    - cls: filtered
      +      args:
      +        storage_conf:
      +          cls: remote
      +          args:
      +            url: http://saam.internal.softwareheritage.org:5003/
      +        filters_conf:
      +        - type: readonly
       storage:
         cls: retry
*******************************************
*** End octocatalog-diff on vangogh.euwest.azure.internal.softwareheritage.org

The staging vault uses also the objstorage from staging:

$ bin/octocatalog-diff --to staging vault.internal.staging.swh.network
...
diff origin/production/vault.internal.staging.swh.network current/vault.internal.staging.swh.network
*******************************************
  File[/etc/softwareheritage/vault/server.yml] =>
   parameters =>
     content =>
      @@ -1,3 +1,10 @@
       ---
      +objstorage:
      +  cls: filtered
      +  storage_conf:
      +    cls: remote
      +    url: http://storage1.internal.staging.swh.network:5003/
      +  filters_conf:
      +  - type: readonly
       storage:
         cls: retry
*******************************************
*** End octocatalog-diff on vault.internal.staging.swh.network

Vault cooker configuration does not change (worker11.euwest, worker1.staging, ...):

$ bin/octocatalog-diff --to staging worker11.euwest.azure
...
*** Running octocatalog-diff on host worker11.euwest.azure.internal.softwareheritage.org
I, [2021-06-28T14:18:54.629481 #29934]  INFO -- : Catalogs compiled for worker11.euwest.azure.internal.softwareheritage.org
I, [2021-06-28T14:18:55.503144 #29934]  INFO -- : Diffs computed for worker11.euwest.azure.internal.softwareheritage.org
I, [2021-06-28T14:18:55.503185 #29934]  INFO -- : No differences
*** End octocatalog-diff on worker11.euwest.azure.internal.softwareheritage.org

Migrated from D5933 (view on Phabricator)

Merge request reports