Skip to content
Snippets Groups Projects
Commit 4d939a59 authored by David Douard's avatar David Douard
Browse files

Use code-block:: console in place of bash 

This allows the generated snippet (by pygnents) to better handle the
selection for copy/pasting. Namely, the prompt will not be selected
which is what is expected for this kind of example snip.
parent d043af4d
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
docs/sysadm/mirror-operations/docker.rst
+ 56
− 56
View file @ 4d939a59
...@@ -16,9 +16,9 @@ the `docker stack deploy ...@@ -16,9 +16,9 @@ the `docker stack deploy
<https://docs.docker.com/engine/reference/commandline/stack_deploy/>`_ command, <https://docs.docker.com/engine/reference/commandline/stack_deploy/>`_ command,
e.g.: e.g.:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker node ls swh:~/swh-mirror$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
py47518uzdb94y2sb5yjurj22 host2 Ready Active 18.09.7 py47518uzdb94y2sb5yjurj22 host2 Ready Active 18.09.7
n9mfw08gys0dmvg5j2bb4j2m7 * host1 Ready Active Leader 18.09.7 n9mfw08gys0dmvg5j2bb4j2m7 * host1 Ready Active Leader 18.09.7
...@@ -54,9 +54,9 @@ the objstorage (thus put them in :file:`/var/lib/docker/volumes`). ...@@ -54,9 +54,9 @@ the objstorage (thus put them in :file:`/var/lib/docker/volumes`).
you should create the docker volumes before starting the docker service. For you should create the docker volumes before starting the docker service. For
example, the ``objstorage`` service uses a volume named ``<STACK>_objstorage``: example, the ``objstorage`` service uses a volume named ``<STACK>_objstorage``:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker volume create -d local \ swh:~/swh-mirror$ docker volume create -d local \
--opt type=none \ --opt type=none \
--opt o=bind \ --opt o=bind \
--opt device=/data/docker/swh-objstorage \ --opt device=/data/docker/swh-objstorage \
...@@ -88,9 +88,9 @@ pinned to specific nodes using labels named ...@@ -88,9 +88,9 @@ pinned to specific nodes using labels named
When you create a local volume for a given container, you should add the relevant label When you create a local volume for a given container, you should add the relevant label
to the docker swarm node metadata with: to the docker swarm node metadata with:
.. code-block:: bash .. code-block:: console
docker node update \ swh:~/swh-mirror$ docker node update \
--label-add org.softwareheritage.mirror.volumes.objstorage=true \ --label-add org.softwareheritage.mirror.volumes.objstorage=true \
<node_name> <node_name>
...@@ -102,18 +102,18 @@ The monitoring services, ``prometheus``, ``prometheus-statsd-exporter`` and ...@@ -102,18 +102,18 @@ The monitoring services, ``prometheus``, ``prometheus-statsd-exporter`` and
``org.softwareheritage.mirror.monitoring``. So make sure to add this label to ``org.softwareheritage.mirror.monitoring``. So make sure to add this label to
one (and only one) node of the cluster: one (and only one) node of the cluster:
.. code-block:: bash .. code-block:: console
docker node update \ swh:~/swh-mirror$ docker node update \
--label-add org.softwareheritage.mirror.monitoring=true \ --label-add org.softwareheritage.mirror.monitoring=true \
<node_name> <node_name>
To check labels defined on a specific node, one can use the ``docker node To check labels defined on a specific node, one can use the ``docker node
inspect`` command: inspect`` command:
.. code-block:: bash .. code-block:: console
docker node inspect \ swh:~/swh-mirror$ docker node inspect \
-f '{{ .ID }} [{{ .Description.Hostname}}]: \ -f '{{ .ID }} [{{ .Description.Hostname}}]: \
{{ range $k, $v := .Spec.Labels }}{{ $k }}={{ $v }} {{ range $k, $v := .Spec.Labels }}{{ $k }}={{ $v }}
{{end}}' <node_name> {{end}}' <node_name>
...@@ -148,9 +148,9 @@ Namely, you need to create a ``secret`` for: ...@@ -148,9 +148,9 @@ Namely, you need to create a ``secret`` for:
For example: For example:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ xkcdpass -d- | docker secret create swh-mirror-db-postgres-password - swh:~/swh-mirror$ xkcdpass -d- | docker secret create swh-mirror-db-postgres-password -
[...] [...]
...@@ -159,10 +159,10 @@ Spawning the swh base services ...@@ -159,10 +159,10 @@ Spawning the swh base services
If you haven't done it yet, clone this git repository: If you haven't done it yet, clone this git repository:
.. code-block:: bash .. code-block:: console
~$ git clone https://gitlab.softwareheritage.org/swh/infra/swh-mirror.git swh:~$ git clone https://gitlab.softwareheritage.org/swh/infra/swh-mirror.git
~$ cd swh-mirror swh:~$ cd swh-mirror
This repository provides the docker compose/stack manifests to deploy all the relevant This repository provides the docker compose/stack manifests to deploy all the relevant
services. services.
...@@ -177,16 +177,16 @@ services. ...@@ -177,16 +177,16 @@ services.
To specify the tag to be used, simply set the :envvar:`SWH_IMAGE_TAG` To specify the tag to be used, simply set the :envvar:`SWH_IMAGE_TAG`
environment variable: environment variable:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ export SWH_IMAGE_TAG=20211022-121751 swh:~/swh-mirror$ export SWH_IMAGE_TAG=20211022-121751
Make sure you have node labels attributed properly. Then you can spawn the Make sure you have node labels attributed properly. Then you can spawn the
base services using the following command: base services using the following command:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker stack deploy -c mirror.yml swh swh:~/swh-mirror$ docker stack deploy -c mirror.yml swh
Creating network swh_default Creating network swh_default
Creating config swh_content-replayer Creating config swh_content-replayer
...@@ -216,7 +216,7 @@ base services using the following command: ...@@ -216,7 +216,7 @@ base services using the following command:
Creating service swh_memcache Creating service swh_memcache
Creating service swh_grafana Creating service swh_grafana
~/swh-mirror$ docker service ls swh:~/swh-mirror$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS ID NAME MODE REPLICAS IMAGE PORTS
ptlhzue025zm swh_content-replayer replicated 0/0 softwareheritage/replayer:20220225-101454 ptlhzue025zm swh_content-replayer replicated 0/0 softwareheritage/replayer:20220225-101454
...@@ -276,18 +276,18 @@ objects. The usual method to update a config in a service is: ...@@ -276,18 +276,18 @@ objects. The usual method to update a config in a service is:
For example, if you edit the file :file:`conf/storage.yml`: For example, if you edit the file :file:`conf/storage.yml`:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker config create storage-2 conf/storage.yml swh:~/swh-mirror$ docker config create storage-2 conf/storage.yml
h0m8jvsacvpl71zdcq3wnud6c h0m8jvsacvpl71zdcq3wnud6c
~/swh-mirror$ docker service update \ swh:~/swh-mirror$ docker service update \
--config-rm storage \ --config-rm storage \
--config-add source=storage-2,target=/etc/softwareheritage/config.yml \ --config-add source=storage-2,target=/etc/softwareheritage/config.yml \
swh_storage swh_storage
swh_storage swh_storage
overall progress: 2 out of 2 tasks overall progress: 2 out of 2 tasks
verify: Service converged verify: Service converged
~/swh-mirror$ docker config rm storage swh:~/swh-mirror$ docker config rm storage
.. Warning:: this procedure will update the live configuration of the service .. Warning:: this procedure will update the live configuration of the service
stack, which will then be out of sync with the stack described in stack, which will then be out of sync with the stack described in
...@@ -306,15 +306,15 @@ how to use the config system in a docker swarm cluster. ...@@ -306,15 +306,15 @@ how to use the config system in a docker swarm cluster.
Note that the :command:`docker service update` command can be used for many other Note that the :command:`docker service update` command can be used for many other
things, for example it can be used to change the debug level of a service: things, for example it can be used to change the debug level of a service:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service update --env-add LOG_LEVEL=DEBUG swh_storage swh:~/swh-mirror$ docker service update --env-add LOG_LEVEL=DEBUG swh_storage
Then you can revert to the previous setup using: Then you can revert to the previous setup using:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service update --rollback swh_storage swh:~/swh-mirror$ docker service update --rollback swh_storage
See the documentation of the `swh service update See the documentation of the `swh service update
<https://docs.docker.com/engine/reference/commandline/service_update/>`_ <https://docs.docker.com/engine/reference/commandline/service_update/>`_
...@@ -332,10 +332,10 @@ services at once. ...@@ -332,10 +332,10 @@ services at once.
This can be done as follow: This can be done as follow:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ export SWH_IMAGE_TAG=<new version> swh:~/swh-mirror$ export SWH_IMAGE_TAG=<new version>
~/swh-mirror$ docker stack deploy -c base-services.yml swh swh:~/swh-mirror$ docker stack deploy -c base-services.yml swh
Note that this will reset the replicas config to their default values. Note that this will reset the replicas config to their default values.
...@@ -343,9 +343,9 @@ Note that this will reset the replicas config to their default values. ...@@ -343,9 +343,9 @@ Note that this will reset the replicas config to their default values.
If you want to update only a specific service, you can also use (here for a If you want to update only a specific service, you can also use (here for a
replayer service): replayer service):
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service update --image \ swh:~/swh-mirror$ docker service update --image \
softwareheritage/replayer:${SWH_IMAGE_TAG} \ softwareheritage/replayer:${SWH_IMAGE_TAG} \
swh_graph-replayer swh_graph-replayer
...@@ -355,13 +355,13 @@ replayer service): ...@@ -355,13 +355,13 @@ replayer service):
script. So we strongly recommend you scale the service back to one before script. So we strongly recommend you scale the service back to one before
updating the image: updating the image:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service scale swh_storage=1 swh:~/swh-mirror$ docker service scale swh_storage=1
~/swh-mirror$ docker service update --image \ swh:~/swh-mirror$ docker service update --image \
softwareheritage/base:${SWH_IMAGE_TAG} \ softwareheritage/base:${SWH_IMAGE_TAG} \
swh_storage swh_storage
~/swh-mirror$ docker service scale swh_storage=16 swh:~/swh-mirror$ docker service scale swh_storage=16
Set up the mirroring components Set up the mirroring components
...@@ -401,20 +401,20 @@ check/update are: ...@@ -401,20 +401,20 @@ check/update are:
Then you need to update the configuration, as described above: Then you need to update the configuration, as described above:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker config create swh_graph-replayer-2 conf/graph-replayer.yml swh:~/swh-mirror$ docker config create swh_graph-replayer-2 conf/graph-replayer.yml
~/swh-mirror$ docker service update \ swh:~/swh-mirror$ docker service update \
--config-rm swh_graph-replayer \ --config-rm swh_graph-replayer \
--config-add source=swh_graph-replayer-2,target=/etc/softwareheritage/config.yml \ --config-add source=swh_graph-replayer-2,target=/etc/softwareheritage/config.yml \
swh_graph-replayer swh_graph-replayer
and and
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker config create swh_content-replayer-2 conf/content-replayer.yml swh:~/swh-mirror$ docker config create swh_content-replayer-2 conf/content-replayer.yml
~/swh-mirror$ docker service update \ swh:~/swh-mirror$ docker service update \
--config-rm swh_content-replayer \ --config-rm swh_content-replayer \
--config-add source=swh_content-replayer-2,target=/etc/softwareheritage/config.yml \ --config-add source=swh_content-replayer-2,target=/etc/softwareheritage/config.yml \
swh_content-replayer swh_content-replayer
...@@ -425,15 +425,15 @@ Graph replayer ...@@ -425,15 +425,15 @@ Graph replayer
To run the graph replayer component of a mirror is just a matter of scaling its service: To run the graph replayer component of a mirror is just a matter of scaling its service:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service scale swh_graph-replayer=1 swh:~/swh-mirror$ docker service scale swh_graph-replayer=1
You can check everything is running with: You can check everything is running with:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service ps swh_graph-replayer swh:~/swh-mirror$ docker service ps swh_graph-replayer
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ioyt34ok118a swh_graph-replayer.1 softwareheritage/replayer:20220225-101454 node1 Running Running 17 minutes ago ioyt34ok118a swh_graph-replayer.1 softwareheritage/replayer:20220225-101454 node1 Running Running 17 minutes ago
...@@ -441,16 +441,16 @@ You can check everything is running with: ...@@ -441,16 +441,16 @@ You can check everything is running with:
If everything is OK, you should have your mirror filling. Check docker logs: If everything is OK, you should have your mirror filling. Check docker logs:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service logs swh_graph-replayer swh:~/swh-mirror$ docker service logs swh_graph-replayer
[...] [...]
or: or:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service logs --tail 100 --follow swh_graph-replayer swh:~/swh-mirror$ docker service logs --tail 100 --follow swh_graph-replayer
[...] [...]
...@@ -459,9 +459,9 @@ Content replayer ...@@ -459,9 +459,9 @@ Content replayer
Similarly, to run the content replayer: Similarly, to run the content replayer:
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service scale swh_content-replayer=1 swh:~/swh-mirror$ docker service scale swh_content-replayer=1
Getting your deployment production-ready Getting your deployment production-ready
...@@ -473,10 +473,10 @@ docker-stack scaling ...@@ -473,10 +473,10 @@ docker-stack scaling
Once the replayer services have been checked, started and are working Once the replayer services have been checked, started and are working
properly, you can increase the replication to speed up the replication process. properly, you can increase the replication to speed up the replication process.
.. code-block:: bash .. code-block:: console
~/swh-mirror$ docker service scale swh_graph-replayer=64 swh:~/swh-mirror$ docker service scale swh_graph-replayer=64
~/swh-mirror$ docker service scale swh_content-replayer=64 swh:~/swh-mirror$ docker service scale swh_content-replayer=64
A proper replication factor value will depend on your infrastructure A proper replication factor value will depend on your infrastructure
capabilities and needs to be adjusted watching the load of the core services capabilities and needs to be adjusted watching the load of the core services
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment