Skip to content
Snippets Groups Projects
Verified Commit 4aaa957e authored by Antoine R. Dumont's avatar Antoine R. Dumont
Browse files

upgrade-swh-service: Make reference names unique

As reference namespace is global, we need to make an extra effort to name
reference uniquely.
parent 3daccff1
No related branches found
No related tags found
No related merge requests found
.. _upgrade-swh-service: .. _deployment-upgrade-swh-service:
Upgrade swh service Upgrade swh service
=================== ===================
...@@ -27,7 +27,7 @@ Then follows the actual :ref:`deployment with debian packaging ...@@ -27,7 +27,7 @@ Then follows the actual :ref:`deployment with debian packaging
<deployment-with-debian-packaging>`. It concludes with the :ref:`deployment with <deployment-with-debian-packaging>`. It concludes with the :ref:`deployment with
kubernetes<deployment-with-kubernetes>` chapter. kubernetes<deployment-with-kubernetes>` chapter.
.. _distinct-services: .. _deployment-upgrade-swh-service-distinct-services:
Distinct Services Distinct Services
----------------- -----------------
...@@ -38,7 +38,7 @@ Distinct Services ...@@ -38,7 +38,7 @@ Distinct Services
- rpc services (scheduler, objstorage, storage, web, ...) - rpc services (scheduler, objstorage, storage, web, ...)
- journal client services (search, scheduler, indexer) - journal client services (search, scheduler, indexer)
.. _code-and-publish-a-release: .. _deployment-upgrade-swh-service-code-and-publish-a-release:
Code and publish a release Code and publish a release
...@@ -50,7 +50,7 @@ Code an evolution or a bugfix in the impacted git repository (usually the master ...@@ -50,7 +50,7 @@ Code an evolution or a bugfix in the impacted git repository (usually the master
branch). Open a diff for review. Land it when accepted. And then release it following branch). Open a diff for review. Land it when accepted. And then release it following
the :ref:`tag and push <tag-and-push>` part. the :ref:`tag and push <tag-and-push>` part.
.. _tag-and-push: .. _deployment-upgrade-swh-service-tag-and-push:
Tag and push Tag and push
~~~~~~~~~~~~ ~~~~~~~~~~~~
...@@ -63,7 +63,7 @@ When ready, `git tag` and `git push` the new tag of the module. Then let jenkins ...@@ -63,7 +63,7 @@ When ready, `git tag` and `git push` the new tag of the module. Then let jenkins
$ git tag -a vA.B.C # (optionally) `git tag -a -s` to sign the tag too $ git tag -a vA.B.C # (optionally) `git tag -a -s` to sign the tag too
$ git push origin --follow-tags $ git push origin --follow-tags
.. _publish-artifacts: .. _deployment-upgrade-swh-service-publish-artifacts:
Publish artifacts Publish artifacts
~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~
...@@ -73,16 +73,18 @@ the tag just pushed). It then builds the debian package and pushes it to our :re ...@@ -73,16 +73,18 @@ the tag just pushed). It then builds the debian package and pushes it to our :re
debian repositories <howto-debian-packaging>`. debian repositories <howto-debian-packaging>`.
.. _troubleshoot: .. _deployment-upgrade-swh-service-troubleshoot:
Troubleshoot Troubleshoot
~~~~~~~~~~~~ ~~~~~~~~~~~~
If jenkins fails for some reason, fix the module be it :ref:`python code If jenkins fails for some reason, fix the module be it :ref:`python code
<code-and-publish-a-release>` or the :ref:`debian packaging <troubleshoot-debian-package>`. <deployment-upgrade-swh-service-code-and-publish-a-release>` or the
:ref:`debian packaging
<deployment-upgrade-swh-service-troubleshoot-debian-package>`.
.. _deployment-with-debian-packaging: .. _deployment-upgrade-swh-service-with-debian-packaging:
Deployment with debian packaging Deployment with debian packaging
...@@ -90,7 +92,7 @@ Deployment with debian packaging ...@@ -90,7 +92,7 @@ Deployment with debian packaging
This mostly involves deploying new version of debian packages to static nodes. This mostly involves deploying new version of debian packages to static nodes.
.. _upgrade-services: .. _deployment-upgrade-swh-service-upgrade-services:
Upgrade services Upgrade services
~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~
...@@ -122,7 +124,7 @@ nodes through the following group names: ...@@ -122,7 +124,7 @@ nodes through the following group names:
See :ref:`deploy-new-lister` for a practical example. See :ref:`deploy-new-lister` for a practical example.
.. _troubleshoot-debian-package: .. _deployment-upgrade-swh-service-troubleshoot-debian-package:
Debian package troubleshoot Debian package troubleshoot
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -144,7 +146,7 @@ Lather, rinse, repeat until it's all green! ...@@ -144,7 +146,7 @@ Lather, rinse, repeat until it's all green!
Deploy Deploy
------ ------
.. _nominal-case: .. _deployment-upgrade-swh-service-nominal-case:
Nominal case Nominal case
~~~~~~~~~~~~ ~~~~~~~~~~~~
...@@ -171,7 +173,7 @@ command, something like: ...@@ -171,7 +173,7 @@ command, something like:
[3] pergamon is already *clush* configured to allow multiple ssh connections in parallel [3] pergamon is already *clush* configured to allow multiple ssh connections in parallel
on our managed infrastructure nodes. on our managed infrastructure nodes.
.. _configuration-change-required: .. _deployment-upgrade-swh-service-configuration-change-required:
Configuration change required Configuration change required
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -187,7 +189,7 @@ Or force a puppet run: ...@@ -187,7 +189,7 @@ Or force a puppet run:
Note: *-t* is not optional Note: *-t* is not optional
.. _long-standing-upgrade: .. _deployment-upgrade-swh-service-long-standing-upgrade:
Long-standing upgrade Long-standing upgrade
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
...@@ -218,7 +220,7 @@ Then: ...@@ -218,7 +220,7 @@ Then:
$ sudo clush -b -w @swh-workers 'systemctl start cron.service; puppet agent --enable' $ sudo clush -b -w @swh-workers 'systemctl start cron.service; puppet agent --enable'
.. _deployment-with-kubernetes: .. _deployment-upgrade-swh-service-with-kubernetes:
Deployment with Kubernetes Deployment with Kubernetes
-------------------------- --------------------------
...@@ -227,20 +229,21 @@ This new deployment involves docker images which are exposing script/services wh ...@@ -227,20 +229,21 @@ This new deployment involves docker images which are exposing script/services wh
running in a virtual python frozen environment. Those versioned images are then running in a virtual python frozen environment. Those versioned images are then
referenced in a specific helm chart which is deployed in a kubernetes rancher cluster. referenced in a specific helm chart which is deployed in a kubernetes rancher cluster.
That cluster runs on machines nodes (with :ref:`specific labels <labels-on-nodes>`) onto which are scheduled pods That cluster runs on machines nodes (with :ref:`specific labels
with containers inside. Those containers are the ones spawning the docker image as <deployment-upgrade-swh-service-labels-on-nodes>`) onto which are scheduled
applications. pods with containers inside. Those containers are the ones spawning the docker
image as applications.
Those docker images are built out of a declared Dockerfile in the `swh-apps`_ Those docker images are built out of a declared Dockerfile in the `swh-apps`_
repository. repository.
You can either: You can either:
- :ref:`Add a new application<add-new-swh-application>` - :ref:`Add a new application<deployment-upgrade-swh-service-add-new-swh-application>`
- :ref:`Update an application<update-swh-application>` - :ref:`Update an application<deployment-upgrade-swh-service-update-swh-application>`
- :ref:`Release a new version of an application<build-and-publish-docker-image-app>` - :ref:`Release a new version of an application<deployment-upgrade-swh-service-build-and-publish-docker-image-app>`
.. _add-new-swh-application: .. _deployment-upgrade-swh-service-add-new-swh-application:
Add new swh application Add new swh application
~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~
...@@ -258,7 +261,7 @@ It's time to build and publish a docker image. It's a multiple steps process tha ...@@ -258,7 +261,7 @@ It's time to build and publish a docker image. It's a multiple steps process tha
executed locally starting with the :ref:`frozen set of dependencies requirements to executed locally starting with the :ref:`frozen set of dependencies requirements to
generate <update-app-frozen-requirements>`. generate <update-app-frozen-requirements>`.
.. _update-swh-application: .. _deployment-upgrade-swh-service-update-swh-application:
Update swh application Update swh application
~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~
...@@ -273,9 +276,9 @@ a minimal and it may be that such change should happen upstream in the swh modul ...@@ -273,9 +276,9 @@ a minimal and it may be that such change should happen upstream in the swh modul
instead. instead.
Once your update is done, commit and push the change, then :ref:`build and publish the Once your update is done, commit and push the change, then :ref:`build and publish the
new docker image <build-and-publish-docker-image-app>`. new docker image <deployment-upgrade-swh-service-build-and-publish-docker-image-app>`.
.. _build-and-publish-docker-image-app: .. _deployment-upgrade-swh-service-build-and-publish-docker-image-app:
Build and publish docker image (recommended) Build and publish docker image (recommended)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -285,10 +288,11 @@ Use the `dedicated jenkins job ...@@ -285,10 +288,11 @@ Use the `dedicated jenkins job
to update the app's frozen requirements, build the docker image with that set and to update the app's frozen requirements, build the docker image with that set and
publish that image to the swh gitlab registry. publish that image to the swh gitlab registry.
Once the application image is published in the registry, you need to :ref:`update the Once the application image is published in the registry, you need to
impacted chart <update-impacted-chart>`. :ref:`update the impacted chart
<deployment-upgrade-swh-service-update-impacted-chart>`.
.. _update-impacted-chart: .. _deployment-upgrade-swh-service-update-impacted-chart:
Update impacted chart Update impacted chart
~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
...@@ -301,7 +305,7 @@ Check that the nodes are properly labelled to receive the application. Then :ref ...@@ -301,7 +305,7 @@ Check that the nodes are properly labelled to receive the application. Then :ref
<argocd-config>` will be in charge of deploying the changes in a rolling upgrade <argocd-config>` will be in charge of deploying the changes in a rolling upgrade
fashion. fashion.
.. _update-app-frozen-requirements: .. _deployment-upgrade-swh-service-update-app-frozen-requirements:
Update app's frozen requirements Update app's frozen requirements
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...@@ -323,15 +327,16 @@ $APP_NAME (e.g. *loader_{git, svn, cvs, ...}*, *lister*, *indexer* ...): ...@@ -323,15 +327,16 @@ $APP_NAME (e.g. *loader_{git, svn, cvs, ...}*, *lister*, *indexer* ...):
$ docker run --rm -v $PWD:/src app-manager generate-frozen-requirements $APP_NAME $ docker run --rm -v $PWD:/src app-manager generate-frozen-requirements $APP_NAME
You have built your frozen requirements that can be committed. Next, we will You have built your frozen requirements that can be committed. Next, we will
:ref:`generate the image updated with that frozen environment <generate-image>`. :ref:`generate the image updated with that frozen environment
<deployment-upgrade-swh-service-generate-image>`.
.. _generate-image: .. _deployment-upgrade-swh-service-generate-image:
Generate image Generate image
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
Build the docker image with the frozen environment and then :ref:`publish it Build the docker image with the frozen environment and then :ref:`publish it
<publish-image>`: <deployment-upgrade-swh-service-publish-image>`:
.. code:: .. code::
...@@ -355,7 +360,7 @@ You must have a gitlab account and generate a personal access token with at leas ...@@ -355,7 +360,7 @@ You must have a gitlab account and generate a personal access token with at leas
`write` access to the `gitlab registry `write` access to the `gitlab registry
<https://gitlab.softwareheritage.org/swh/infra/swh-apps/container_registry/>`_. <https://gitlab.softwareheritage.org/swh/infra/swh-apps/container_registry/>`_.
.. _publish-image: .. _deployment-upgrade-swh-service-publish-image:
Publish image Publish image
~~~~~~~~~~~~~ ~~~~~~~~~~~~~
...@@ -370,19 +375,21 @@ then push the image: ...@@ -370,19 +375,21 @@ then push the image:
$ docker push $FULL_IMAGE $ docker push $FULL_IMAGE
$ docker push $FULL_IMAGE_LATEST $ docker push $FULL_IMAGE_LATEST
Do not forget to :ref:`commit the changes and tag <commit-changes-and-tag>`. Do not forget to :ref:`commit the changes and tag
<deployment-upgrade-swh-service-commit-changes-and-tag>`.
Finally, let's :ref:`update the impacted chart <update-impacted-chart>` with the new Finally, let's :ref:`update the impacted chart
docker image version. <deployment-upgrade-swh-service-update-impacted-chart>` with the new docker
image version.
.. _commit-changes-and-tag: .. _deployment-upgrade-swh-service-commit-changes-and-tag:
Commit and tag Commit and tag
~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~
Commit and tag the changes. Commit and tag the changes.
.. _labels-on-nodes: .. _deployment-upgrade-swh-service-labels-on-nodes:
Labels on nodes Labels on nodes
~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment