Out of memory conditions make the rancher nodes unresponsive
We've had repeated crashes of the rancher metal nodes lately. Every time, this is associated with a memory usage spike.
I'm guessing this is caused by the way we're using swap as a "tmpfs with disk fallback" crutch; at some point, a process goes awry and swap usage makes the system slow enough that it can't be OOM-killed.
We should review whether the tmpfs + swap trick is really still needed, and consider dropping our usage of swap if it's not anymore.