docker: Fix journal client services

Services acting as journal clients were always exiting prematurely due to not created kafka topics they were trying to subscribe to.

So ensure needed topics will be created when starting the kafka service and wait for their creation before starting the journal client services.

As a consequence, this makes a call to docker-compose up -d command a little bit longer before it returns.

Also fix a service configuration and enable DEBUG logging in another one (see commit log).

---

Migrated from D5090 (view on Phabricator)

mentioned in merge request !92 (closed)

awesome, thanks.

Merge request was accepted

approved this merge request

Merge request was merged

closed

Please note a recent version of docker-compose is needed to allow the environment to start with the healthcheck keywork, It was not working with the version 1.26.2 but is ok with 1.28.2

❯ docker-compose version
docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

❯ docker-compose -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml up -d --force-recreate --no-dep swh-search
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.kafka: 'healthcheck'
services.swh-scheduler-journal-client.depends_on contains an invalid type, it should be an array
services.swh-indexer-journal-client.depends_on contains an invalid type, it should be an array

❯ docker-compose version
docker-compose version 1.28.2, build 67630359
docker-py version: 4.4.1
CPython version: 3.7.9
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019


❯ docker-compose -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Recreating docker_nginx_1                        ... done
...

(forgot to mention: Thanks, it's really a nice improvement)

! In !91 (closed), @vsellier wrote: Please note a recent version of docker-compose is needed to allow the environment to start with the healthcheck keywork, It was not working with the version 1.26.2 but is ok with 1.28.2

❯ docker-compose version
docker-compose version 1.26.2, build eefe0d31
docker-py version: 4.2.2
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019

❯ docker-compose -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml up -d --force-recreate --no-dep swh-search
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.kafka: 'healthcheck'
services.swh-scheduler-journal-client.depends_on contains an invalid type, it should be an array
services.swh-indexer-journal-client.depends_on contains an invalid type, it should be an array

❯ docker-compose version
docker-compose version 1.28.2, build 67630359
docker-py version: 4.4.1
CPython version: 3.7.9
OpenSSL version: OpenSSL 1.1.0l  10 Sep 2019


❯ docker-compose -f docker-compose.yml -f docker-compose.search.yml -f docker-compose.override.yml up -d
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Recreating docker_nginx_1                        ... done
...

Ah right, the feature was introduced in docker-compose 1.27.0 (see commit).

We need to find a way to restrict the version of docker-compose to use then, maybe by changing the version field value at top of docker-compose.yml file.

Ah right, the feature was introduced in docker-compose 1.27.0

Actually, healthcheck it is a feature introduced in version 2.1 of Compose file format.

Bumping version to 2.1 in compose files seems to fix the observed issue (just tested with docker-compose 1.25.0 packaged in debian buster).