staging infra: Reproduce existing production setup in a compact way

Dependency order to follow

• gateway (routing)
• swh-storage (db, swh-storage service)
• swh-objstorage (disks, swh-objstorage service)
• swh-indexer storage (db, service)
• swh-web
• swh-scheduler (db, rabbitmq, swh-scheduler services)
• swh-deposit (db, service)
• 1 worker with at least 1 loader (worker0 & worker1 with loader-git)
• swh-vault
• update workers with checker/loader deposit
• update workers with 1 lister (forge=gitlab, instance=inria)
• update workers with 1 indexer
• Make icinga checks green on the expected origins (parmap, cpython, etc...)

Note:

• Should be a matter of creating the right branch ('staging' for example) in swh-site's repository
• calling the right environment when `puppet agent --test --noop --environment='staging'
• production code should only be modified if there are issues identified (there was ;)
• in the end the staging branch should be merged into production (without any impact on production, that's somehow has been my implicit plan)

Plan: From provisioning node (creation in our hypervisor) to delegating puppet for the configuration/installation services For some of our services, this is ambitious:

• postgres: puppetized the db creation and user done
• postgres: bootstrap db schema (-> this one needs work upfront in our different services to have a common use interface) [1]
• rabbitmq: server installation with users setup
• rabbitmq: improve the configuration option to be in sync with our current instance (migrated/migration$490)

Tests:

• from webapp: save code now -> browsing ok

• push a deposit via deposit client cli -> browsing in webapp ok

• from webapp: request vault cooking -> download in webapp ok

• listing one forge and see new origins from that forge [2]

• origin-content-metadata running

• [1] I consider this out of scope (i need to draw a line somewhere ;) Also we as a team started this, that needs to be finalized IIRC.

• [2] https://webapp.internal.staging.swh.network/browse/search/?q=gitlab.inria.fr&with_visit&with_content

---

Migrated from T1875 (view on Phabricator)