How to Use Varnish and Cloudflare for Maximum Caching

This article is part of a series on building a sample application --- a multi-image gallery blog --- for performance benchmarking and optimizations. (View the repo here.)

As we can see in this report, our site's landing page loads very quickly and generally scores well, but it could use another layer of caching and even a CDN to really do well.

To learn more about GTMetrix and other tools you can use to measure and debug performance, see Improving Page Load Performance: Pingdom, YSlow and GTmetrix

Let's use what we've learned in our previous Varnish post, along with the knowledge gained in the Intro to CDN and Cloudflare posts to really tune up our server's content delivery now.

Varnish

Varnish was created solely for the purpose of being a type of super-cache in front of a regular server.

Note: Given that Nginx itself is a pretty good server already, people usually opt for one or the other, not both. There's no harm in having both, but one does have to be wary of cache-busting problems which can occur. It's important to set them both up properly so that the cache of one of them doesn't remain stale while the other's is fresh. This can lead to different content being shown to different visitors at different times. Setting this up is a bit outside the context of this post, and will be covered in a future guide.

We can install Varnish by executing the following:

curl -L https://packagecloud.io/varnishcache/varnish5/gpgkey | sudo apt-key add -
sudo apt-get update
sudo apt-get install -y apt-transport-https

The current list of repos for Ubuntu does not have Varnish 5+ available, so additional repositories are required. If the file /etc/apt/sources.list.d/varnishcache_varnish5.list doesn't exist, create it. Add to it the following:

deb https://packagecloud.io/varnishcache/varnish5/ubuntu/ xenial main
deb-src https://packagecloud.io/varnishcache/varnish5/ubuntu/ xenial main

Then, run:

sudo apt-get update
sudo apt-get install varnish
varnishd -V

The result should be something like:

$ varnishd -V
varnishd (varnish-5.2.1 revision 67e562482)
Copyright (c) 2006 Verdens Gang AS
Copyright (c) 2006-2015 Varnish Software AS

We then change the server's default port to 8080. We're doing this because Varnish will be sitting on port 80 instead, and forwarding requests to 8080 as needed. If you're developing locally on Homestead Improved as instructed at the beginning of this series, the file you need to edit will be in /etc/nginx/sites-available/homestead.app. Otherwise, it'll probably be in /etc/nginx/sites-available/default.

server {
    listen 8080 default_server;
    listen [::]:8080 default_server ipv6only=on;

Next up, we'll configure Varnish itself by editing /etc/default/varnish and replacing the default port on the first line (6081) with 80:

DAEMON_OPTS="-a :80 \
   -T localhost:6082 \
   -f /etc/varnish/default.vcl \
   -S /etc/varnish/secret \
   -s malloc,256m"

The same needs to be done in /lib/systemd/system/varnish.service:

ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Finally, we can restart both Varnish and Nginx for the changes to take effect:

sudo service nginx restart
sudo /etc/init.d/varnish stop
sudo /etc/init.d/varnish start
systemctl daemon-reload

The last command is there so that the previously edited varnish.service daemon settings also reload, otherwise it'll only take into account the /etc/default/varnish file's changes. The start + stop procedure is necessary for Varnish because of a current bug which doesn't release ports properly unless done this way.

Comparing the result with the previous one, we can see that the difference is minimal due to the landing page already being extremely optimized.

Sidenote

Both of the low grades are mainly the result of us "not serving from a consistent URL", as GTMetrix would put it:

This happens because we used random images to populate our galleries, and the sample of randomness was small, so some of them are repeated. This is fine and won't be an issue when the site is in production. In fact, this is one of the very rare cases where a site will by default score better in production than it does in development.

The post How to Use Varnish and Cloudflare for Maximum Caching appeared first on SitePoint.