Memcached seems to be automatically cleared pretty often on moma
On moma
, the server where the production webapp is hosted, there is an automatic run of puppet on a regular basis.
When puppet runs on moma, we can see that the memcached configuration is updated each time which will trigger a refresh of the memcached service and thus clear the current webapp cache.
*** swh-deploy: starting test run on moma.internal.softwareheritage.org...
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Applying configuration version '1638972115'
Notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/content:
--- /etc/memcached.conf 2021-12-08 13:35:21.435550583 +0000
+++ /tmp/puppet-file20211208-3773096-k2mx4 2021-12-08 14:02:09.969214287 +0000
@@ -10,7 +10,7 @@
logfile /var/log/memcached.log
# Use <num> MB memory max to use for object storage.
--m 1235
+-m 1176
Notice: /Stage[main]/Memcached/File[/etc/memcached.conf]/content: current_value '{md5}6cd8b2c619c53eae5f6bff188e35ccf2', should be '{md5}83b44ab40734309b38aa6d55d877f106' (noop)
Info: /Stage[main]/Memcached/File[/etc/memcached.conf]: Scheduling refresh of Service[memcached]
Notice: /Stage[main]/Memcached/Service[memcached]: Would have triggered 'refresh' from 1 event
Notice: Class[Memcached]: Would have triggered 'refresh' from 2 events
Notice: Stage[main]: Would have triggered 'refresh' from 1 event
Notice: Applied catalog in 8.81 seconds
Connection to moma.internal.softwareheritage.org closed.
This is problematic for the authentication feature of the webapp as OpenID Connect profiles of users are stored in the django cache backed by memcached.
As a consequence, when the memcached is cleared users are automatically logout and they must click on the login link again to authenticate again, which is not really convenient.
We should either stop touching the memcached config when running puppet or disable the automatic puppet run on moma to fix that issue.
Migrated from T3779 (view on Phabricator)