varnish: use different backends per vhosts (!245) · Merge requests · Platform / Infrastructure / Puppet / puppet-swh-site

Antoine R. Dumont requested to merge generated-differential-D4440-source into generated-differential-D4440-target Nov 06, 2020

Related to T2747

Test Plan

bin/octocatalog moma [1]
- drops default varnish backend
- installs the deposit and webapp backends (different configuration, same behavior)
bin/octocatalog webapp0:
- drops default varnish backend
- installs the varnish webapp backend (different conf, same behavior)
bin/octocatalog webapp.internal.staging.swh.network: do nothing (no reverse proxy there)
bin/octocatalog deposit.internal.staging.swh.network: do nothing (no rp there)
bin/octocatalog rp0.internal.staging.swh.network: installs the varnish backends (deposit, webapp targetting the respective staging nodes)

And vagrant:

vagrant up staging-rp0
vagrant up staging-deposit
vagrant up staging-webapp

Then tamper with /etc/hosts (local and within the staging-rp0) so we can use the same internal staging fqdn to check if the reverse proxying work (it does ;)

bin/octocatalog-diff --octocatalog-diff-args --no-truncate-details --to T2747_rp moma
Found host moma.softwareheritage.org
WARN     -> Environment "open-template1" contained non-word characters, correcting name to open_template1
WARN     -> Environment "wip-pg-hba-rules-in-yaml" contained non-word characters, correcting name to wip_pg_hba_rules_in_yaml
Cloning into '/tmp/swh-ocd.zLll8wDl/environments/production/data/private'...
done.
Cloning into '/tmp/swh-ocd.zLll8wDl/environments/T2747_rp/data/private'...
done.
*** Running octocatalog-diff on host moma.softwareheritage.org
I, [2020-11-06T14:57:07.661026 #29964]  INFO -- : Catalogs compiled for moma.softwareheritage.org
I, [2020-11-06T14:57:10.012042 #29964]  INFO -- : Diffs computed for moma.softwareheritage.org
diff origin/production/moma.softwareheritage.org current/moma.softwareheritage.org
*******************************************
- Concat::Fragment[/etc/varnish/includes.vcl:backend_default]
*******************************************
+ Concat::Fragment[/etc/varnish/includes.vcl:deposit] =>
   parameters =>
      "content": "include \"includes/01_deposit.vcl\";"
      "order": "01"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat::Fragment[/etc/varnish/includes.vcl:webapp] =>
   parameters =>
      "content": "include \"includes/01_webapp.vcl\";"
      "order": "01"
      "target": "/etc/varnish/includes.vcl"
*******************************************
- Concat_fragment[/etc/varnish/includes.vcl:backend_default]
*******************************************
+ Concat_fragment[/etc/varnish/includes.vcl:deposit] =>
   parameters =>
      "content": "include \"includes/01_deposit.vcl\";"
      "order": "01"
      "tag": "_etc_varnish_includes.vcl"
      "target": "/etc/varnish/includes.vcl"
*******************************************
+ Concat_fragment[/etc/varnish/includes.vcl:webapp] =>
   parameters =>
      "content": "include \"includes/01_webapp.vcl\";"
      "order": "01"
      "tag": "_etc_varnish_includes.vcl"
      "target": "/etc/varnish/includes.vcl"
*******************************************
- File[/etc/varnish/includes/01_backend_default.vcl]
*******************************************
+ File[/etc/varnish/includes/01_deposit.vcl] =>
   parameters =>
      "group": "root"
      "mode": "0644"
      "notify": "Exec[vcl_reload]"
      "owner": "root"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend deposit
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
+ File[/etc/varnish/includes/01_webapp.vcl] =>
   parameters =>
      "group": "root"
      "mode": "0644"
      "notify": "Exec[vcl_reload]"
      "owner": "root"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend webapp
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
  File[/etc/varnish/includes/50_vhost_archive.softwareheritage.org.vcl] =>
   parameters =>
     content =>
      @@ -17,4 +17,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = webapp;
               }
           }
*******************************************
  File[/etc/varnish/includes/50_vhost_deposit.softwareheritage.org.vcl] =>
   parameters =>
     content =>
      @@ -15,4 +15,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = deposit;
               }
           }
*******************************************
- Profile::Varnish::Vcl_include[backend_default]
*******************************************
+ Profile::Varnish::Vcl_include[deposit] =>
   parameters =>
      "basename": "deposit"
      "order": "01"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend deposit
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
  Profile::Varnish::Vcl_include[vhost_archive.softwareheritage.org] =>
   parameters =>
     content =>
      @@ -17,4 +17,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = webapp;
               }
           }
*******************************************
  Profile::Varnish::Vcl_include[vhost_deposit.softwareheritage.org] =>
   parameters =>
     content =>
      @@ -15,4 +15,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = deposit;
               }
           }
*******************************************
+ Profile::Varnish::Vcl_include[webapp] =>
   parameters =>
      "basename": "webapp"
      "order": "01"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend webapp
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
  Profile::Varnish::Vhost[archive.softwareheritage.org] =>
   parameters =>
     backend_http_host =>
      + ::1
     backend_http_port =>
      + 9080
     backend_name =>
      + webapp
*******************************************
  Profile::Varnish::Vhost[deposit.softwareheritage.org] =>
   parameters =>
     backend_http_host =>
      + ::1
     backend_http_port =>
      + 9080
     backend_name =>
      + deposit
*******************************************
- Varnish::Vcl[/etc/varnish/includes/01_backend_default.vcl]
*******************************************
+ Varnish::Vcl[/etc/varnish/includes/01_deposit.vcl] =>
   parameters =>
      "file": "/etc/varnish/includes/01_deposit.vcl"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend deposit
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
+ Varnish::Vcl[/etc/varnish/includes/01_webapp.vcl] =>
   parameters =>
      "file": "/etc/varnish/includes/01_webapp.vcl"
      "content": >>>
# backend_default.vcl
#
# Default backend definition.
#
# File managed by puppet. All modifications will be lost.

backend webapp
 {
    .host = "::1";
    .port = "9080";
}
<<<
*******************************************
  Varnish::Vcl[/etc/varnish/includes/50_vhost_archive.softwareheritage.org.vcl] =>
   parameters =>
     content =>
      @@ -17,4 +17,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = webapp;
               }
           }
*******************************************
  Varnish::Vcl[/etc/varnish/includes/50_vhost_deposit.softwareheritage.org.vcl] =>
   parameters =>
     content =>
      @@ -15,4 +15,5 @@
               } else {
                   set req.http.X_FORWARDED_PROTO = "https";
      +            set req.backend_hint = deposit;
               }
           }
*******************************************
*** End octocatalog-diff on moma.softwareheritage.org

Migrated from D4440 (view on Phabricator)

varnish: use different backends per vhosts

Test Plan

Merge request reports