[k8s staging] Some pods are regularly evicted or in error
kubectl --context archive-staging-rke2 get pods -o wide | grep -v -e Running -e Pending -e Completed 12:25:56
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
graphql-6dcb5c5b84-cclv5 0/1 Evicted 0 3m4s <none> rancher-node-staging-rke2-worker2 <none> <none>
graphql-6dcb5c5b84-xjhb5 0/1 Evicted 0 5d1h <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-all-585787d8cc-ldkg9 0/1 ContainerStatusUnknown 0 10m 10.42.233.177 rancher-node-staging-rke2-worker2 <none> <none>
lister-bower-654c4896d5-4xrr7 0/1 Evicted 0 46m <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-bower-654c4896d5-wlwhr 0/1 Evicted 0 11m <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-golang-5c546c757-qk69t 0/1 Evicted 0 24h <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-launchpad-548db5d5-d78bj 0/1 Error 0 12m 10.42.18.240 rancher-node-staging-rke2-worker3 <none> <none>
lister-launchpad-548db5d5-ndg5w 0/1 Evicted 0 25m <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-launchpad-548db5d5-nzjxh 0/1 Error 0 11m 10.42.233.190 rancher-node-staging-rke2-worker2 <none> <none>
lister-opam-579d4b4599-jhns8 0/1 Evicted 0 11m <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-opam-579d4b4599-zqfr6 0/1 Evicted 0 44m <none> rancher-node-staging-rke2-worker2 <none> <none>
lister-opam-697c57db55-fbwjt 0/1 Evicted 0 2d <none> rancher-node-staging-rke2-worker2 <none> <none>
loader-addforgenow-78bf56fd45-f9tj6 0/1 Error 0 18h 10.42.233.156 rancher-node-staging-rke2-worker2 <none> <none>
loader-bzr-6b6679854-w9cqd 0/1 Evicted 0 10d <none> rancher-node-staging-rke2-worker2 <none> <none>
loader-debian-776b5f47ff-rbfgg 0/1 Evicted 0 42m <none> rancher-node-staging-rke2-worker2 <none> <none>
loader-pypi-66f95595cb-lvqr5 0/1 Evicted 0 58m <none> rancher-node-staging-rke2-worker2 <none> <none>
loader-svn-64ff4d6d79-8dg77 0/1 Evicted 0 10d <none> rancher-node-staging-rke2-worker2 <none> <none>It seems related to disk pressure on the nodes:
40m Warning Evicted pod/loader-pypi-66f95595cb-lvqr5 The node was low on resource: ephemeral-storage. Container loaders was using 650560000, which exceeds its request of 0.