Add Varnish to an Elastic Beanstalk Environment Running WordPress

Updated Jan 25, 2018

This post is part of the series WordPress and Elastic Beanstalk.

Varnish

Varnish is an Apache service that can improve your site’s performance by serving cached files. When a user requests a file from your site, Varnish checks its cache; if it finds a cached version of the file it returns that file, else it forwards the request to the server as usual and potentially caches the response.

Varnish & WordPress

Varnish can lead to big performance gains for a WordPress site in particular, since WordPress isn’t serving any static files that can be cached by the browser. Instead, when request hits the server for a webpage, WordPress has to load, find the appropriate template, and run a bunch of PHP to service the request. With Varnish installed, when a request first comes in for a page WordPress renders it but Varnish stores the finished product; for subsequent requests for the same page Varnish can just return the cached file, bypassing WordPress entirely.

We don’t want to cache all files, however, as some return dynamic content based on the request: in particular, we don’t want to cache files when a user is logged in. Fortunately Varnish allows for configuration to do just that.

Another consideration are WordPress’s use of nonce fields. A nonce is valid for 12 hours, and, despite its name, can be used more than once. If you configure Varnish to cache pages for less than 12 hours you should be fine. In the below configuration we specify a 1-hour caching period.

Elastic Beanstalk configuration

Note that a bad configuration can take your site down. To properly configure Varnish we have to tell it to listen on port 80, where httpd usually listens, and tell httpd to listen on another port where we forward requests. If the Varnish configuration isn’t working, requests to port 80 will return 503 errors. Ideally you’d test Varnish on a staging site first to ensure proper configuration.

Add all of the following configuration files to your .ebextensions directory. The files could be combined into a single file but are broken out here for explanatory purposes.

Varnish installation

varnish-a-install.config Download Copy


    
    packages:
      yum:
        varnish: []

The above simply installs Varnish on your instances. This tutorial assumes you are running Amazon Linux 2017.09, which supports Varnish 3.0.7.

Varnish ulimits

varnish-b-ulimits.config Download Copy


    
    files:
      "/etc/security/limits.d/varnish.conf":
        mode: "000444"
        owner: root
        group: root
        content: |
          # Varnish requires certain ulimits. If we don't define these then Varnish
          # won't start.
          ec2-user hard nofile 131072
          ec2-user soft nofile 131072
          ec2-user hard memlock 82000
          ec2-user soft memlock 82000
          ec2-user hard nproc unlimited
          ec2-user soft nproc unlimited

Varnish requires minimum ulimits for the number of files it can create, its memory allocation, and its number of processes. If we don’t update these Varnish will complain and won’t start.

Varnish configuration

varnish-c-vcl.config Download Copy


    
    files:
      "/etc/varnish/default.vcl":
        mode: "000444"
        owner: root
        group: root
        content: |
    
          # HOST AND PORT.
          ##########################################################################
          backend default {
            .host = "127.0.0.1";
            .port = "81";
          }
    
          # RECV FUNCTION
      ##########################################################################
          sub vcl_recv {
    
            # IF PURGE REQUEST CHECK PURGE TOKEN.
            if (req.request == "PURGE") {
              if (req.http.X-Purge-Token == "XXX") {
                return (lookup);
              }
              error 405 "Not allowed";
            }
    
            # PIPE ALL NON-STANDARD REQUESTS.
            if (req.request != "GET" &&
              req.request != "HEAD" &&
              req.request != "PUT" &&
              req.request != "POST" &&
              req.request != "TRACE" &&
              req.request != "OPTIONS" &&
              req.request != "DELETE"
            ) {
              return (pipe);
            }
    
            # ONLY CACHE GET AND HEAD REQUESTS.
            if (req.request != "GET" && req.request != "HEAD") {
              return (pass);
            }
    
            # FORWARD EVERYTHING TO WWW AT HTTPS TO AVOID CACHING HTTP REQUESTS.
            if ((req.http.host ~ "^(?i)(www\.)example.com" && req.http.X-Forwarded-Proto !~ "(?i)https") ||
              req.http.host ~ "^(?i)example.com"
            ) {
              set req.http.X-Redir-Url = "https://www.example.com" + req.url;
              error 750 req.http.X-Redir-Url;
            }
    
            # NEVER CACHE ADMIN OR LOGIN.
            if (req.url ~ "^/wp-(admin|login|cron)") {
              return (pass);
            }
    
            # REMOVE THE WP TEST COOKIE.
            set req.http.Cookie = regsuball(req.http.Cookie, "wordpress_test_cookie=[^;]+(; )?", "");
    
            # DON'T CACHE IF USER LOGGED IN.
            if (req.http.Cookie ~ "wordpress_") {
              return (pass);
            }
    
            # REMOVE ALL CLIENT COOKIES.
            unset req.http.Cookie;
    
            # IF WE MADE IT HERE WE WANT TO SERVE CACHED FILE.
            return (lookup);
          }
    
          # HIT FUNCTION
          ##########################################################################
          sub vcl_hit {
            if (req.request == "PURGE") {
              purge;
              error 200 "Purged";
            }
          }
    
          # MISS FUNCTION
          ##########################################################################
          sub vcl_miss {
            if (req.request == "PURGE") {
              purge;
              error 200 "Purged";
            }
          }
    
          # FETCH FUNCTION
          ##########################################################################
          sub vcl_fetch {
            set beresp.ttl = 1h;
          }
    
          # DELIVER FUNCTION
          ##########################################################################
          sub vcl_deliver {
            if (obj.hits > 0) {
              set resp.http.X-Cache = "HIT";
            }
            else {
              set resp.http.X-Cache = "MISS";
            }
          }
    
          # ERROR FUNCTION
          ##########################################################################
          sub vcl_error {
            if (obj.status == 750) {
              set obj.http.Location = obj.response;
              set obj.status = 301;
              return (deliver);
            }
          }

This is the Varnish configuration file, written using the Varnish Configuration Language (VCL).

Line 2 Varnish will look for a configuration file at this location.
Lines 3-5 Set permissions and user and group for the configuration file.
Lines 10-13 Of note is line 12, where we tell Varnish to forward requests to port 81. Later we’ll instruct httpd to listen on this port.
Line 17 The vcl_recv function handles requests. This is where we make modifications and decide what to do with various requests.
Lines 20-25 Varnish provides a method to purge files from the cache if need be. For instance, if you update a post and want the new version immediately available you could send a PURGE request to your server at the address of the post. This request will only be honored if it includes a custom header X-Purge-Token with the value “XXX”. Change the “XXX” to your own secret value so third parties can’t purge pages from your site.
Lines 28-37 Varnish includes a special “pipe” command that we should use for non-standard requests. This pipes the request to our server.
Lines 40-42 Only cache GET and HEAD requests.
Lines 45-50 This assumes you’ve completed both Add a Custom Domain and Force HTTPS tutorials. Replace “example.com” with your domain. If the request is not over HTTPS or lacks “www” we redirect to “https://www.example.com”. The custom error 750 is handled later on.
Lines 53-55 We don’t want to cache any of the admin or login pages.
Line 58 Remove the WordPress test cookie so we don’t match it below when checking whether user is logged in.
Lines 61-63 Don’t cache objects if the user is logged in.
Line 66 Remove all client cookies from request.
Line 69 Return object from cache if available else from backend server.
Lines 74-79 Handle purge request on cache hit.
Lines 83-88 Handle purge request on cache miss.
Lines 92-94 Cache objects for 1 hour.
Lines 98-105 Set custom header so we can see if returned page was a hit (cached) or miss (not cached).
Lines 109-115 Handle our custom “error” to return a redirect that we defined in lines 45-50.

Note that there may be additional pages that you don’t want to cache. For instance, if you’re using WooCommerce you’ll want to exclude at least the cart and checkout pages from caching. You can achieve this by adding additional rules to the vcl_recv function.

Port configuration

varnish-d-ports.config Download Copy


    
    commands:
      010_httpd.conf:
        command: "sed -i 's/Listen 8080/Listen 80/g' /etc/httpd/conf/httpd.conf"
      011_httpd.conf:
        command: "sed -i 's/Listen 80/Listen 81/g' /etc/httpd/conf/httpd.conf"
      040_varnish:
        command: "sed -i 's/VARNISH_LISTEN_PORT=6081/VARNISH_LISTEN_PORT=80/g' /etc/sysconfig/varnish"
      041_varnish:
        command: "sed -i 's/VARNISH_ADMIN_LISTEN_PORT=6082/VARNISH_ADMIN_LISTEN_PORT=2000/g' /etc/sysconfig/varnish"

Lines 2-5 Modify httpd.conf to tell httpd to listen on port 81.
Lines 6-7 Modify the configuration to tell varnish to listen on port 80.
Lines 8-9 Modify the configuration to tell varnish admin to listen on port 2000.

Starting the service

varnish-e-restart.config Download Copy


    
    commands:
      create_post_dir:
        command: "mkdir /opt/elasticbeanstalk/hooks/appdeploy/post"
        ignoreErrors: true
    files:
      "/opt/elasticbeanstalk/hooks/appdeploy/post/99_restart_varnish.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
          #!usr/bin/env bash
          sudo service varnish restart

The above is based on this post from junkheap.net. It makes use of the special /opt/elasticbeanstalk/hooks/appdeploy/post directory by adding a simple shell script that restarts Varnish.

Next steps

When you re-deploy to your environment you should have Varnish up and running. You can check its behavior by looking for the X-Cache response headers. Remember that if you’re logged in Varnish will not cache pages. If you have questions or suggestions please feel free to leave comments below.

Comments

No comments exist. Be the first!