Monitoring // Tony Metzidis

Writing Custom Metrics to Stackdriver in Golang

Instrumentation is a critical part of any application. Along with system counters like cpu, heap, free disk, etc– it’s important to create application-level metrics to make sure health is measured closer to your customer’s experience.

Example metrics could be user-registration, password-change, profile-change, etc. If you see a major spike or dip in these metrics, a wider problem could be indicated.

For this example a custom metric was needed, and no infrastructure was in place for harvesting it (e.g. collectd). Golang is handy for creating an easy-to-install daemon which performs the measurement and periodically harvests the data into stackdriver.

Feb 13, 2019

3 min read

Gcp Stackdriver Monitoring Tutorial

Delegating Admin Credentials using IAM Roles and Cloudwatch Alerts

It’s hard to strike the right balance with admin rights–either the rights are too strict and people can’t get work done or they’re too lenient and you have security issues.

As a compromise, AWS provides the AssumeRole feature which lets admins temporarily escalate their role to perform a task.

It’s important when setting this up that you alert the team when it’s used. Here we’ll talk about how to set up the roles, give teams access to the roles and create an alert system when the roles are assumed.

Dec 12, 2015

2 min read

Aws Cloudwatch Alerts Monitoring

On Software Scaffolding

waterloo_bridge_1815 A new lightrail line is being built in my city with bridges passing over the major boulevards. Seeing the elaborate scaffolding evoked comparisons to software engineering. What does scaffolding look like in software? Does software need to be erected like a bridge via scaffolding? Without a doubt: yes.

Here are some elements of software “scaffolding”:

Error log instrumentation with a formal error log schema (i.e. errors are uniquely identifiable in a MECE schema)
Operational instrumentation with reports , dashboards and alerts
Performance profiling on methods, database calls, rest calls, system calls and any blocking IO.
Client-side performance instrumentation and sampling of the population.
Benchmarks established for datasource SLA, server SLA and client time-to-interactivity, among others.
Unit tests
Functional Tests
Client-side tracking of passive (view), positive ( click, message, follow) and negative (cancel, close-window) behaviors

This scaffolding gives you the resolution needed to understand your application and your audience. Moreover, it empowers you make well-informed decisions and the agility to move quickly.

Jul 9, 2015

1 min read

Monitoring Software