-
IPV6 Migration Guide for Developers using AWS EC2 -- A Primer
With the news that AWS will be now charging about $4 / instance-month for public IPv4 addresses, many developers who procrastinated ipv6 migration are finally updating both ends of their development setup.
It’s a great time to migrate, as all the intermediate infrastructure now supports IPV6 readily. Moreover, you’ll benefit from permanent , global addresses for your development instances.
Pros
- A single, global, stable address for EC2 instances that never changes. No need for dynamic DNS and other hacks
- No need to pay for Elastic IP addresses on dev instances
- Global addressing for mutual duplex services (no more NAT needed)
- Better flexibility and clarity for addressing, including Link Local & local addresses
Cons
- Time needed to migrate infra to IPV6
- Clumsier & less-memorable addresses, with unfamiliar idioms (e.g. no more using 127.0.0.1 or 192.168.1.1 – though there are replacements) inherent
- Bugs in legacy code that assumes 32 bit & string-representations of ipv4 addresses
Concepts & Approach
In theory, IPV6 uses 128bit addresses in place of 32 bit. Most of the intermediate infra (ISP, backbone) is now compatible. The two areas of attention for developers would be the server side with AWS , and the client side with your home/office network.
-
Free SSL Certificates using ACM (AWS Certificate Manager)
2016 may be the year of free SSL, and AWS ACM (AWS Certificate Manager) is a great offering for Cloudfront & ELB users (most web apps).
Not only is it free, but it’s also the simplest certificate management platform
- request a new certificate in minutes
- no server config needed
- no certificate , chain or private key management
- automatic certificate rotation
Here’s how to create a certificate and then install it onto your cloudfront distribution.
-
Creating a Varnish Load Balancer for Opsworks
Varnish is an amazing platform – it can easily help you handle 100x traffic and is easy to add to your existing frontend or API layer with little to no change to your app.
Here we’ll go over some neat tricks leveraging chef, the AWS Opsworks API and the opsworks
configurelifecycle event to create a lighting fast load balancer & reverse proxy that automatically updates itself.Setup
-
Create a new
varnishlayer that installs thevarnishandjqpackages -
Using the AWS EC2 Container Registry with EC2 Container Service
AWS announced recently that it’s EC2 Container Registry (ECR) is now available. ECR simplifies hosting private images. Previously, you had to manually push your docker.io credentials to each EC2 instance – likely a deliberate pain-point encouraging you to use ECR. With ECR, EC2 container hosts can easily fetch private images using IAM authentication.
Here are some of the gotchyas and stumbling blocks to help you get your repository up quickly and painlessly.
-
Securing Your Network Using Auto-Updating Security Groups
We all know that no ports should be open to the internet for development purposes, but for convenience it’s common to find a security group with port 22 (SSH) open to
0.0.0.0/0. Even narrower ingress rules can create backdoors.Here we’ll show you how to create an auto-updating security group that adds your active WAN IP address when you connect. This way, only your active IP is authorized.
Create the “development” security group with no ingress
aws ec2 create-security-group --group-name=development --group-description="ssh access for my dev machine"Create a limited role that can only update this security group
Since you may want to embed this script on your router or elsewhere, it’s important to generate a restricted access key that can only do one thing Create a user and add this policy
-
Using AWS Lambda for Web Video Transcoding
Often your creative team will produce master videos in 4k or 1080p, but you need to downcode these videos into 720p/1080p for web broadcasting. Here we automate transcoding of masters into web-friendly formats like 720p h264 mp4 & webm.
AWS Elastic Transcoder is a cloud video transcoding service. At it’s simplest it transcodes video files from one bitrate, framerate, codec, container, etc–into another. By default you trigger new jobs either manually in the aws console or via the rest API. And naturally all inputs & outputs are saved in S3.
-
Wordpress Cron on Opsworks
By default Wordpress uses it’s own pseudo-cron which triggers with every request. Obviously this is wasteful since (a) the queue needs to be inspected with every GET and (b) jobs like publishing articles will interfere with serving content.
Some suggest calling the
wp-cron.phpGET request with curl in a cron like this* * * * * curl http://www.mysite.com/wp-cron.phpbut that’s sub-optimal since it needlessly ties up a worker during the cron execution.
-
HTTP Redirects with Cloudfront & S3
Redirects can account for a significant share of direct traffic so taking a few minutes to optimize them is worthwhile.
Using Cloudfront & S3 for redirects will improve responsiveness, reduce server load and improve management (since they are managed via
aws-clior the console).Let’s say you have a typical
.htaccessredirect like this.RewriteEngine On ### re-direct to www RewriteCond %{HTTP_HOST} !^www\. [NC] RewriteRule ^(.*)$ http://www.%{HTTP_HOST}%{REQUEST_URI} [R=301,L]Or worse, it could look like this in your
index.php -
Opsworks -- Quickly Listing Hosts on the Command Line
Here’s a great example of using the aws-cli to speed up your life. Uses jq and aws-cli
# bash / zsh function function opsworks-hosts-prod () { aws opsworks describe-instances --stack-id=fffff-fffff-ffff-fff-fffffff | jq '.Instances[].PublicDns' | grep -v null | sed s/\"//g } # usage $ opsworks-hosts-prod XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com XXXXX.compute-1.amazonaws.com -
On Software Scaffolding
A new lightrail line is being built in my city with bridges passing over the major boulevards. Seeing the elaborate scaffolding evoked comparisons to software engineering. What does scaffolding look like in software? Does software need to be erected like a bridge via scaffolding? Without a doubt: yes.Here are some elements of software “scaffolding”:
- Error log instrumentation with a formal error log schema (i.e. errors are uniquely identifiable in a MECE schema)
- Operational instrumentation with reports , dashboards and alerts
- Performance profiling on methods, database calls, rest calls, system calls and any blocking IO.
- Client-side performance instrumentation and sampling of the population.
- Benchmarks established for datasource SLA, server SLA and client time-to-interactivity, among others.
- Unit tests
- Functional Tests
- Client-side tracking of passive (view), positive ( click, message, follow) and negative (cancel, close-window) behaviors
This scaffolding gives you the resolution needed to understand your application and your audience. Moreover, it empowers you make well-informed decisions and the agility to move quickly.
-
Opsworks before-migrate.rb
Opsworks is a convenient, powerful and free service provided by AWS to simplify the management of EC2 nodes. The real power of the system is exposed through customizing various stages of the instance lifecycle by creating custom-tailored chef-solo recipes.
While Amazon provides a powerful deployment layer for PHP applications, it stops short once the PHP code has been checked out of git. For Laravel or other composer apps, you’ll have to customize your deployment. The most elegant and straightforward method is through custom deployment hooks. Here’s how to build a before_migration.rb script to build a Laravel app.
-
Tony Metzidis
Software Developer