Creating a Varnish Load Balancer for Opsworks

Posted on Fri 15 January 2016 in aws

Varnish is an amazing platform -- it can easily help you handle 100x traffic and is easy to add to your existing frontend or API layer with little to no change to your app.

Here we'll go over some neat tricks leveraging chef, the AWS Opsworks API and the opsworks configure lifecycle event to create a lighting fast load balancer & reverse proxy that automatically updates itself.

Setup

  1. Create a new varnish layer that installs the varnish and jq packages

  2. Activate custom cookbooks. It's easiest to just use s3 deployments so you don't need a separate git repo.

The varnish::backends recipe

This recipe uses the aws opsworks describe-instances API to automatically update the backend list. The cookbook's templates set up the proper health checks for each backend.

Make sure you set default[:varnish][:backend_layer] and default[:varnish][:backend_region] to specify the opsworks layer of your origin servers.

# install the default.vcl that references the backends. this is included in the cookbook templates
cookbook_file "/etc/varnish/default.vcl" do
  source "default.vcl"
end

# make sure these two are set to your environment before running
if(! (node[:varnish][:backend_layer] && node[:varnish][:backend_region]))
  Chef::Log.debug("node[varnish][backend_layer] and backend_region needs to be set")
  return;
end
# call the aws api to fetch the active instance internal DNS names
instances = %x(aws --region=#{node[:varnish][:backend_region]} opsworks describe-instances --layer-id=#{node[:varnish][:backend_layer]} |jq '.Instances[]'.PrivateDns).gsub(/["]/, '').split("\n")


if(instances.count < 1)
  Chef::Log.error("No instances found. No changes made")
  return
end

template "/etc/varnish/backends.vcl" do
  Chef::Log.debug("Activating these varnish backends:  [#{instances.join(' ')}]")
  source "backends.vcl.erb"
  owner 'root'
  group 'root'
  mode 0644
  variables(
    :instances => instances,
  )
end

execute "reload vcl config" do
  command "/usr/sbin/varnish_reload_vcl"
end

backends.vcl.erb

Here's the template for the backends. This sets up the backends and health check specs. the gsub accounts for naming limitations in vcl files

<% @instances.each do |instance| %>
backend <%= instance.gsub(/[\.-]/, '_') %>  {
    .host = "<%= instance %>";
    .port = "80";
    .probe = {
         .url = "/";
         .interval = 10s;
         .timeout = 5 s;
         .window = 5;
         .threshold = 3;
    }
}
<% end %>
director origin round-robin{
<% @instances.each do |instance| %>
    {
    .backend=<%= instance.gsub(/[\.-]/, '_') %>;
    }
  <% end %>
}

More Info

  • Get familiar with the "configure" lifecycle event. This triggers whenever your instances are started and stopped
  • Varnish round-robin directors -- varnish can manage backend health checks and distribute load among an array of origin nodes. We'll focus on V3.x since it's easier to set up via Amazon linux AMI and configuration docs are more common.
  • The Opsworks DescribeInstances API which can be used to list the instance IP addresses for a given layer.

scaling infrastructure opsworks varnish chef