Deploy swh-indexer v2.9.1 on staging and prod

tag v2.8.0
Tagger: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 23 08:57:20 2022 +0100

v2.8.0

* journal writer: only flush kafka once per batch
* origin_head: Do not fetch complete snapshots for non-FTP visits
* ExtrinsicMetadataIndexer: Add support for metadata with origin in context

This only requires resetting offsets of the indexer consumer on swh.journal.objects.raw_extrinsic_metadata, so it can pick up metadata from deposits.

tag v2.9.0
Tagger: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Tue Nov 29 15:28:28 2022 +0100

v2.9.0

* storage: Insert from temporary tables in consistent order
* Drop content_language and content_ctags tables and related SQL functions

tag v2.9.1
Tagger: Valentin Lorentz <vlorentz@softwareheritage.org>
Date:   Wed Nov 30 11:46:40 2022 +0100

v2.9.1

* Fix ordering and idempotence in the 136 -> 137 upgrade script
* docs: Remove remaining references to ctags and content_language
* Fix 'Invalid IPv6 URL' crash
* Fix crash when indexing two REMD objects from the same deposit

Checklist:

staging:

• staging: stop indexer journal clients
• staging: storage backend schema migrations
• staging: update storage backend service
• staging: reset raw_extrinsic_metadata offsets for reindexing
• staging: restart indexer journal clients

prod:

• prod: stop indexer journal clients
• prod: storage backend schema migrations
• prod: update storage backend service
• prod: upgrade indexer journal client [1]
• prod: reset raw_extrinsic_metadata offsets for reindexing
• prod: restart indexer journal clients [2]

[1]

root@pergamon:~# clush -b -w @indexer-workers "dpkg -l python3-swh.indexer*" | grep ii
ii  python3-swh.indexer         2.9.1-1~swh1~bpo10+1 all          Software Heritage Content Indexer
ii  python3-swh.indexer.storage 2.9.1-1~swh1~bpo10+1 all          Software Heritage Content Indexer Storage

[2] https://grafana.softwareheritage.org/goto/qJV7VBFVz?orgId=1

I'll take care of that this week.

Maybe worth landing https://forge.softwareheritage.org/D8873 first?

assigned to @olasd

changed title from Deploy swh-indexer v2.8.0 on staging and prod to Deploy swh-indexer v2.9.0 on staging and prod

changed the description

added state:wip label

mentioned in commit swh-apps@0f7fae40

marked the checklist item staging: stop indexer journal clients as completed

marked the checklist item staging: storage backend schema migrations as completed

marked the checklist item staging: update storage backend service as completed

marked the checklist item staging: reset raw_extrinsic_metadata offsets for reindexing as completed

marked the checklist item staging: restart indexer journal clients as completed

mentioned in commit swh-apps@2d5664a4

changed title from Deploy swh-indexer v2.9.0 on staging and prod to Deploy swh-indexer v2.9.1 on staging and prod

changed the description

assigned to @ardumont and unassigned @olasd

marked the checklist item prod: stop indexer journal clients as completed