Empty and reindex swh.journal.indexed.origin_extrinsic_metadata

The swh.journal.indexed.origin_extrinsic_metadata topic contains invalid dates, which crash the swh-search journal client, so:

• staging: Stop indexer journal client (origin extrinsic metadata) -> argocd

• staging: Stop swh.search @indexed journal client -> search0.internal.staging.swh.network

• staging: Delete topic swh.journal.indexed.origin_extrinsic_metadata

• staging: Recreate topic with proper config (number of partitions, etc.)

• staging: Reset swh.indexer.journal_client.extrinsic_metadata consumer group offsets

• staging: Reset swh.search.journal_client.indexed-v0.11 consumer group offsets for swh.journal.indexed.origin_extrinsic_metadata

• staging: Restart swh.indexer journal client -> argocd

• staging: Restart swh.search journal client -> search0

• staging done

• prod: Stop indexer journal client (origin extrinsic metadata) -> indexer-worker*.euwest.azure

• prod: Stop swh.search @indexed journal client -> search1.internal.softwareheritage.org

• prod: update swh.search storage backend

• prod: update swh.indexer storage backend

• prod: Remove topic swh.journal.indexed.origin_extrinsic_metadata

• prod: Recreate topic with proper config (number of partitions, replicas, etc.)

• prod: Reset swh.indexer.journal_client.extrinsic_metadata consumer group offsets

• prod: Restart swh.indexer journal client -> indexer-worker*.euwest.azure

• prod: Restart swh.search journal client -> search1.internal.softwareheritage.org

• prod done

changed title from Empty topic swh.journal.indexed.origin_extrinsic_metadata to Empty and reindex swh.journal.indexed.origin_extrinsic_metadata

changed the description

changed the description

mentioned in issue #4652 (closed)

assigned to @olasd

changed the description

marked the checklist item staging: Stop swh.search @indexed journal client -> search0.internal.staging.swh.network as completed

mentioned in commit swh-apps@9009d880

marked the checklist item staging: Stop indexer journal client (origin extrinsic metadata) -> argocd as completed

I got a bit side-tracked publishing the docker image for swh-indexer (looks like the gitlab container registry setup hadn't been tested until now).

Back on track now!

marked the checklist item staging: Delete topic swh.journal.indexed.origin_extrinsic_metadata as completed

marked the checklist item staging: Recreate topic with proper config (number of partitions, etc.) as completed

Topic delete:

/opt/kafka/bin/kafka-topics.sh  --bootstrap-server storage1.internal.staging.swh.network:9092 --delete --topic swh.journal.indexed.origin_extrinsic_metadata

Offset reset:

/opt/kafka/bin/kafka-consumer-groups.sh  --bootstrap-server storage1.internal.staging.swh.network:9092 --reset-offsets --to-earliest --group swh.indexer.journal_client.extrinsic_metadata --topic swh.journal.objects.raw_extrinsic_metadata --execute

marked the checklist item staging: Reset swh.indexer.journal_client.extrinsic_metadata consumer group offsets as completed

marked the checklist item staging: Restart swh.indexer journal client -> argocd as completed

changed the description

Offset reset for swh.search:

/opt/kafka/bin/kafka-consumer-groups.sh  --bootstrap-server storage1.internal.staging.swh.network:9092 --reset-offsets --to-earliest --group swh.search.journal_client.indexed-v0.11 --topic swh.journal.indexed.origin_extrinsic_metadata --execute