Allowing Cached HTTPS Traffic From Drupal With Varnish And Pound

Varnish is a web application accelerator that helps to speed up websites. It works by sitting in front of the web server and caching web pages that are served by it.

When a request for a web page is made Varnish passes this request on to the web server, which then responds to the request as it normally would. Varnish then caches the result of this request before sending it to the user. This means that the next time the page is requested the cached version of the page is sent to the user and the web server isn't actually part of the process. Because Varnish doesn't run any website code or access any databases the response is therefore much faster than the original response from the web server. In fact, simply having Varnish in place can provide large speed improvements on sites without much configuration. Varnish is exceptionally good at this job and can speed up sites significantly.

One thing that Varnish doesn't cater for is HTTPS traffic. This is by design and as such if you have a Varnish setup you will need to bypass it entirely if you want to use encrypted HTTPS traffic.

One way around this is to use a reverse proxy system called Pound. Pound sits in front of Varnish on the HTTPS port and decrypts all HTTPS requests before sending them to Varnish. All normal HTTP traffic is ignored by Pound and goes through Varnish and is treated as it normally is.

I'm not going to go through installing and setting up Varnish here as that is a lengthy operation in itself. If you are interested then I have already written a description of the process of installing and configuring Varnish, which also includes a description of configuring Drupal.

To install Pound on a Debian based system just run the following.

sudo apt-get install pound

RedHat based systems are pretty similar, but in this case they use yum.

sudo yum install pound

Out of the box, Pound will not start on system boot, to set this edit the setting in the file /etc/default/pound.

startup=1

Before editing the main Pound configuration we need to generate a certificate file that Pound understands so that it can decrypt our HTTPS transmissions and forward them onto Varnish. If you already have a trusted SSL certificate you may still need to complete this step in order to generate a certificate that Pound is happy with. For testing purposes we are going to use the existing test SSL certificate that comes with Ubuntu to demonstrate how to do this (kept at /etc/ssl/certs/ssl-cert-snakeoil.pem). To generate a certificate file for Pound you need to combine the pem and key files together like this.

mkdir keys
cp /etc/ssl/certs/ssl-cert-snakeoil.pem keys
sudo openssl rsa -in /etc/ssl/private/ssl-cert-snakeoil.key >> keys/ssl-cert-snakeoil.pem

The main Pound configuration file is found at /etc/pound/pound.cfg, which needs to be edited to do two things. As a default Pound will listen to port 80, which is already being listened to by our Varnish server and as such needs to be removed. We then need to add a rule make Pound listen to HTTPS traffic and tell it what to do with the requests.

Your Pound configuration file should look like the following.

User    "www-data"
Group    "www-data"
RootJail  "/var/pound"
LogLevel    1
 
## check backend every X secs:
Alive    30
 
# poundctl control socket
Control "/var/run/pound/poundctl.socket"
 
ListenHTTPS
    Address 10.0.0.1  # put your server's public IP address here
        Port 443
        Cert "/path/to/ssl-cert-snakeoil.pem"
        HeadRemove "X-Forwarded-Proto"
        AddHeader "X-Forwarded-Proto: https"
        Service
            BackEnd
                Address 127.0.0.1
                Port 80
            End
        End
End

The above options set some default settings and then allow Pound to listen to incoming HTTPS requests. We tell Pound to decrypt these requests and then forward them to port 80 as unencrypted traffic. The X-Forwarded-Proto header is set to 'https' so that Varnish and the web application know that it should be dealing with HTTPS traffic. The 'Cert' parameter references the pem file that we generated above, but should be changed to your trusted certificate.

Also note that we aren't adding a 'ListenHTTP' directive here as we don't want Pound to listen to HTTP traffic at all.

If you have already setup your web server to listen to HTTPS traffic then you'll need to turn this off before starting Pound. In Apache this is normally kept in the form of a Listen directive. You can either remove this or just change the port by altering the following configuration directive.

    Listen 44300

Finally, we just need to add a vcl_hash callback to our default.vcl file. This will allow us to differentiate between HTTP and HTTPS traffic when trying to cache pages.

sub vcl_hash {
  hash_data(req.url);
  if (req.http.host) {
    hash_data(req.http.host);
  } else {
    hash_data(server.ip);
  }
  # Use special internal SSL hash for https content
  # X-Forwarded-Proto is set to https by Pound
  if (req.http.X-Forwarded-Proto ~ "https") {
    hash_data(req.http.X-Forwarded-Proto);
  }
  return (hash);
}

With all that in place you should be able to restart Apache, Varnish and Pound to get everything talking together.

sudo service apache2 restart && sudo service varnish restart && sudo service pound restart

You can double check what services are listening to what ports by running 'netstat -nlp', which should produce something like the following. This assumes that the backend port that Varnish is sending traffic to is 8080, which is what Apache is responding on.

$ sudo netstat -nlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
.....
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      11605/varnishd
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      11554/apache2
.....
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      11647/pound
......

If you are having trouble starting Pound then take a look at the Pound log, which will be kept in /var/log/syslog on Debian systems.

sudo tail /var/log/syslog

Here it will tell you about any port conflicts or incorrect SSL certificate files so you should be able to figure out what is wrong and correct it.

There are one or two security considerations to bear in mind here. If you are unencrypting traffic and then sending this traffic over a network to a different server then there is a small chance of that traffic being intercepted. For this reason it is essential that Pound is only used on a private network or on the same machine as the Varnish instance.

Finally, those of you who are following along at home probably realise the elephant in the room here. Although both HTTP and HTTPS anonymous traffic will be cached, authenticated traffic will not be. So if all of your users login to view your site then using Pound to send traffic through Varnish will not reduce the amount of resources you need. The main problem is that you don't really want to serve cached pages to users who have logged in as there will be a chance that they will see the cache that another user has generated. There are a couple of things you can do to allow caching involving setting custom headers and cookies and even ESI (edge side includes).

As a bonus I thought I would create an Ansible playbook that will install Pound alongside an existing Varnish on Ubuntu setup. The two templates used in this playbook (pound_config.j2 and pound_initscript.j2) can be found in the article above.

---
- hosts: all
  tasks:
    - name: install Pound
      apt: pkg=pound state=installed
      sudo: true
 
    - name: setup Pound initscript
      template: src=pound_initscript.j2 dest=/etc/default/pound
      sudo: true
 
    - name: create snakeoil pem certificate directory
      file: path=/home/{{ user }}/keys state=directory
 
    - name: copy snakeoil certificate pem file
      command: cp /etc/ssl/certs/ssl-cert-snakeoil.pem /home/{{ user }}/keys creates=/home/{{ user }}/keys/ssl-cert-snakeoil.pem
 
    - name: generate pem file with key file
      shell: openssl rsa -in /etc/ssl/private/ssl-cert-snakeoil.key >> ssl-cert-snakeoil.pem chdir=/home/{{ user }}/keys creates=/home/{{ user }}/keys/ssl-cert-snakeoil-pound.pem
      sudo: true
 
    - name: add Pound config file
      template: src=pound_config.j2 dest=/etc/pound/pound.cfg
      sudo: true
      notify:
        - restart pound
 
    - name: ensure pound is started
      service: name=pound state=started
      sudo: true
 
    - name: get iptables rules
      shell: iptables -L
      register: iptablesrules
      sudo: true
      always_run: true
      changed_when: False
 
    - name: add apache iptable rule
      command: iptables -I INPUT 1 -p tcp --dport 443 -j ACCEPT -m comment --comment "pound https port"
      sudo: true
      when: iptablesrules.stdout.find("pound") == -1
      notify:
       - save iptables
 
  handlers:
    - name: restart pound
      service: name=pound state=restarted
      sudo: true
 
    - name: save iptables
      command: iptables-save
      sudo: true
      notify:
       - restart ufw
 
    - name: restart ufw
      service: name=ufw state=restarted
      sudo: true