Skip to content

Add an optional swh.storage read replica to the docker setup

Nicolas Dandrimont requested to merge generated-differential-D4030-source into generated-differential-D4030-target

Summary This configuration adds a new swh.storage database and backend using logical postgresql replication. On startup of the read-replica storage backend, the replication is configured between both database hosts.

The configuration also overrides the swh.web and swh.vault containers to use the new read-only replica.

Two extra scaffolding commits:

  • Make sure to explicitly empty .pg_service.conf before adding entries
  • Support overriding the host in wait_pgsql

Depends on !74 (closed) Depends on !75 (closed)

(also requires the changes to swh.storage SQL in swh-storage!571 (closed))

Test Plan

docker-compose -f docker-compose.yml -f docker-compose.storage-read-replica.yml -f docker-compose.override.yml up -d
export SWH_SCHEDULER_URL=http://127.0.0.1:5008/
swh scheduler task add load-git -p oneshot url=https://github.com/Microsoft/MS-DOS

And then clickity click on the web interface to try to cook one of the directories.

Migrated from D4030 (view on Phabricator)

Merge request reports