Skip to content

indexer: Allow journal client authentication configuration

Antoine R. Dumont requested to merge generated-differential-D8492-source into generated-differential-D8492-target

This extends the current production indexer journal clients to use their credentials and avoid passing through the vpn.

This does not touch the staging workers configuration though.

This got adapted according to the documentation [1].

Related to T4459

Test Plan

$ $SWH_PUPPET_ENVIRONMENT_HOME/bin/octocatalog-diff indexer-worker04.euwest.azure.internal.softwareheritage.org
diff origin/production/indexer-worker04.euwest.azure.internal.softwareheritage.org current/indexer-worker04.euwest.azure.internal.softwareheritage.org
*******************************************
  File[/etc/softwareheritage/indexer/extrinsic_metadata.yml] =>
   parameters =>
     content =>
      @@ -100,6 +100,10 @@
         - kafka3.internal.softwareheritage.org
         - kafka4.internal.softwareheritage.org
      -  group_id: swh.indexer.journal_client.extrinsic_metadata
      +  group_id: swh::deploy::indexer_journal_client::journal::username-swh.indexer.journal_client.extrinsic_metadata
         prefix: swh.journal.objects
      +  sasl.mechanism: SCRAM-SHA-512
      +  security.protocol: SASL_SSL
      +  sasl.username: swh::deploy::indexer_journal_client::journal::username
      +  sasl.password: swh-deploy-indexer_journal_client-journal-password
       tools:
         name: swh-metadata-detector
*******************************************
  File[/etc/softwareheritage/indexer/origin_intrinsic_metadata.yml] =>
   parameters =>
     content =>
      @@ -100,7 +100,11 @@
         - kafka3.internal.softwareheritage.org
         - kafka4.internal.softwareheritage.org
      -  group_id: swh.indexer.journal_client.origin_intrinsic_metadata
      +  group_id: swh::deploy::indexer_journal_client::journal::username-swh.indexer.journal_client.origin_intrinsic_metadata
         prefix: swh.journal.objects
         batch_size: 200
      +  sasl.mechanism: SCRAM-SHA-512
      +  security.protocol: SASL_SSL
      +  sasl.username: swh::deploy::indexer_journal_client::journal::username
      +  sasl.password: swh-deploy-indexer_journal_client-journal-password
       tools:
         name: swh-metadata-detector
*******************************************
*** End octocatalog-diff on indexer-worker04.euwest.azure.internal.softwareheritage.org

staging: noop

*** Running octocatalog-diff on host worker0.internal.staging.swh.network
I, [2022-09-15T17:49:45.747441 #3456278]  INFO -- : Catalogs compiled for worker0.internal.staging.swh.network
I, [2022-09-15T17:49:46.142017 #3456278]  INFO -- : Diffs computed for worker0.internal.staging.swh.network
I, [2022-09-15T17:49:46.142058 #3456278]  INFO -- : No differences
*** End octocatalog-diff on worker0.internal.staging.swh.network

Migrated from D8492 (view on Phabricator)

Merge request reports