sysadm/deployment: Add how to run swh stack in a local kube cluster

Refs. swh/infra/sysadm-environment#5344

docs/sysadm/deployment/howto-run-swh-in-local-kube-cluster.rst

+.. _howto-run-swh-in-local-kube-cluster:
+
+How to run the swh stack in a local kubernetes cluster
+======================================================
+
+.. admonition:: Intended audience
+   :class: important
+
+   staff/syadmin members who wants to run a production-like stack swh in a local
+   kubernetes cluster
+
+.. _howto-requirements:
+
+Requirements
+------------
+
+- `helm <https://helm.sh/>`_ the package manager for kubernetes
+- a local kubernetes environment, we assume `kind <https://kind.sigs.k8s.io>`_
+  in this document (another could be `minikube
+  <https://minikube.sigs.k8s.io>`_)
+- one up-to-date local checkout of the `swh charts
+  <https://gitlab.softwareheritage.org/swh/infra/ci-cd/swh-charts.git>`_
+
+Place yourselves in the root folder of your local checkout of swh-charts.
+
+.. _howto-create-cluster:
+
+Create a local cluster
+~~~~~~~~~~~~~~~~~~~~~~
+
+We will first create a local cluster named 'local-cluster' with the necessary
+dependencies for the swh stack to run:
+
+.. code::
+
+   make local-cluster-create
+   make local-cluster-install-deps
+
+Note:
+
+- The associated cluster configuration is written at `$HOME/.kube/config.d/local-cluster.yaml`
+
+.. code-block::
+
+   $ pwd
+   /path/to/swh/sysadm-environment/swh-charts
+   $ make local-cluster-create
+   ---
+   kind: Cluster
+   apiVersion: kind.x-k8s.io/v1alpha4
+   nodes:
+   - role: control-plane
+   - role: worker
+   - role: worker
+   - role: worker
+   Creating cluster "local-cluster" ...
+    ✓ Ensuring node image (kindest/node:v1.30.0)
+    ✓ Preparing nodes
+    ✓ Writing configuration
+    ✓ Starting control-plane
+    ✓ Installing CNI
+    ✓ Installing StorageClass
+    ✓ Joining worker nodes
+   Set kubectl context to "kind-local-cluster"
+   You can now use your cluster with:
+
+   kubectl cluster-info --context kind-local-cluster --kubeconfig
+     $HOME/.kube/config.d/local-cluster.yaml
+
+   $ make local-cluster-install-deps
+   Release "ingress-nginx" does not exist. Installing it now.
+   NAME: ingress-nginx
+   LAST DEPLOYED: Fri Jul  5 10:45:49 2024
+   NAMESPACE: ingress-nginx
+   STATUS: deployed
+   REVISION: 1
+   TEST SUITE: None
+   ...
+   NAME: rabbitmq-operator
+   LAST DEPLOYED: Fri Jul  5 10:46:10 2024
+   NAMESPACE: default
+   STATUS: deployed
+   REVISION: 1
+   TEST SUITE: None
+   NOTES:
+   ...
+   CHART NAME: rabbitmq-cluster-operator
+   CHART VERSION: 4.3.10
+   APP VERSION: 2.9.0
+   +info https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
+   Release "cloudnative-pg" does not exist. Installing it now.
+   NAME: cloudnative-pg
+   LAST DEPLOYED: Fri Jul  5 10:46:12 2024
+   NAMESPACE: cnpg-system
+   STATUS: deployed
+   REVISION: 1
+   TEST SUITE: None
+   NOTES:
+   ...
+   Release "cert-manager" does not exist. Installing it now.
+   NAME: cert-manager
+   LAST DEPLOYED: Fri Jul  5 10:46:14 2024
+   NAMESPACE: cert-manager
+   STATUS: deployed
+   REVISION: 1
+   TEST SUITE: None
+   NOTES:
+   ...
+   Release "k8ssandra-operator" does not exist. Installing it now.
+   NAME: k8ssandra-operator
+   LAST DEPLOYED: Fri Jul  5 10:46:59 2024
+   NAMESPACE: k8ssandra-operator
+   STATUS: deployed
+   REVISION: 1
+   TEST SUITE: None
+
+Install the cluster-components chart
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+We have defined various gitted `local-cluster.yaml` files in our repository
+swh-charts. As most functionalities are toggled off by default, we need to
+enable those when installing the charts.
+
+We'll do this using locally non-gitted override files at the root of the
+repository, namely:
+
+- cluster-components (cc): local-cluster-cc.override.yaml
+- swh: local-cluster-swh.override.yaml
+
+.. code-block::
+
+   # Install the cluster-components chart (mostly backends)
+   $ make cc-local-cluster
+
+   # Give it some time for the various backends to start properly
+   # The first time around, plenty of docker images will get pulled
+   # so that can take a while
+
+   # One can add some kubernetes command line to wait for some condition to be
+   met
+   # kubectl --context kind-local-cluster wait --for=condition=Ready
+   #   pod/search-es-node-0 \
+     --namespace swh
+
+   # Install the swh chart (swh stack)
+   $ make swh-local-cluster
+
+
+Note: The Makefile detects your (optional) override files, ensure they are
+correctly named or nothing will get installed.
+
+.. code-block:: yaml
+
+   $ cat local-cluster-cc.override.yaml
+
+   podPriority:
+     enabled: true
+
+   svix:
+     enabled: true
+
+   rabbitmq:
+     enabled: true
+
+   cloudnativePg:
+     enabled: true
+
+   kafka:
+     enabled: true
+
+   cassandra:
+     enabled: true
+
+   elasticsearch:
+     enabled: true
+
+   redis:
+     enabled: true
+
+   $ cat local-cluster-swh.override.yaml
+
+   storage:
+     enabled: true
+
+   web:
+     enabled: true
+
+   webhooks:
+     enabled: true
+
+   deposit:
+     enabled: true
+
+   toolbox:
+     enabled: true
+
+   scheduler:
+     enabled: true
+
+   cookers:
+     enabled: false
+
+   indexers:
+     enabled: true
+
+   scrubber:
+     enabled: true
+
+   graphql:
+     enabled: false
+
+   listers:
+     enabled: true
+
+   loaders:
+     enabled: true
+
+   loaderMetadata:
+     enabled: true
+
+   checkerDeposit:
+     enabled: true
+
+   memcached:
+     enabled: false
+
+   podPriority:
+     enabled: true
+
+   vault:
+     enabled: true
+
+   indexerStorage:
+     enabled: true
+
+   search:
+     enabled: true
+
+   objstorage:
+     enabled: false
+
+   counters:
+     enabled: true
+
+   alter:
+     enabled: false
+
+   storageReplayer:
+     enabled: true
+
+
+The full local-cluster configuration can be found in their respective values
+files (and you can override more than just the enabled flag):
+
+- `cluster-components/values/local-cluster.yaml <https://gitlab.softwareheritage.org/swh/infra/ci-cd/swh-charts/-/blob/production/cluster-components/values/local-cluster.yaml?ref_type=heads>`_
+- `swh/values/local-cluster.yaml <https://gitlab.softwareheritage.org/swh/infra/ci-cd/swh-charts/-/blob/production/swh/values/local-cluster.yaml?ref_type=heads>`_

docs/sysadm/deployment/index.rst

@@ -15,4 +15,5 @@ SWH Software Deployment
    howto-add-forge-now-process-requests
    howto-add-journal-user-credential
    howto-bulk-ingest
+   howto-run-swh-in-local-kube-cluster
    webhooks-svix