Upgrade the Prometheus setup to Thanos
The main question that needs solving is making sure the prometheus instance doesn't get bogged down by long term storage of metrics. I've been meaning to look into thanos[12] which makes some people whose judgement I trust happy, there's only so many hours in each day.
Migrated from T1362 (view on Phabricator)