production: Migrate rpc counter to dynamic infrastructure

List of services to migrate:

• gunicorn-swh-counters
• swh-counters-journal-client

gunicorn-swh-counters access to:

• counters1: redis backend. Not to be migrated. Stays on that node.
• counter1: /srv/softwareheritage/counters holding *.json historic dataset

journal client access to:

• counter1: redis backend

Plan:

• swh/infra/ci-cd/swh-charts!334 (merged): swh-charts: Deploy journal client to dynamic infra
• Deploy
• Deactivate journal client in counter1
• (manual) counters1: Decommission journal client
• Checks
  • https://grafana.softwareheritage.org/goto/3V4L2M2Sz?orgId=1: grafana board
  • There shall be no lag
• Historic data served by gitlab in swh.counters repository as snippet
• swh/infra/ci-cd/swh-charts!335 (merged): cronjob: Add counter cronjob to refresh periodically cache (conditional).
• swh/infra/ci-cd/swh-charts!336 (closed): swh-charts: Migrate rpc service to dynamic infra
• Adapt redis backend so it can be remotely accessed by the pods in the cluster (if needed) (already done with staging)
• Deploy counters rpc service to dynamic infra
• Adapt firewall rule so workers can access thanos.internal.admin.swh.network
• swh/infra/ci-cd/swh-charts!337 (merged): swh-charts: Adapt dependent services to use the new counters rpc
• (manual) Stop gunicorn-swh-counters
• Checks
• swh/devel/swh-docs!407 (merged): docs: Update service-urls reference [2]

Post-migration:

• swh/infra/puppet/puppet-swh-site!691 (merged): Decommission counter1 swh services, keep only the redis running

[2] https://docs.softwareheritage.org/sysadm/network-architecture/service-urls.html#public-urls-1

Refs. #5179 (closed)

Refs. #5110 (closed)