Autoscaling etcd using SRV discovery on AWS

Acknowledgements Thanks to John 'retr0h' Dewey for releasing the ansible role I borrowed from extensively to make this happen. Also thanks to Kelsey 'kelseyhightower' Hightower for the SetupNetworkEnvironment tool and for generally being a heroic individual in the kubernetes community. Motivation Just over a year ago, I took my first…

A Practical Guide to deploying Netflix's BLESS Certificate Authority

Foreword: Special thanks are due to Ryan Lane and Chris Steipp of lyft, both for authoring their open source bless client and for specific help via Slack when I ran into things I didn't understand. Thank you also to Russell Lewis of Netflix for writing BLESS in the first place!…

YAML in Go: Parsing nested yaml using the ghodss/yaml library

The intent of this article is to shed light on something that, as a newcomer to Golang and a programmer falling somewhere between beginner and intermediate in general, I found very confusing. I refer of course to the titular subject, parsing Yaml for use in a Go application. My need…

AWS CodePipeline - CI, the Amazon Way

I recently began work on an application, written in Golang, that is ultimately intended to replace our MongoDB cluster backup solution. I, being the curious type, decided to try out a few AWS services that I'd never used before. My objective was to setup a simple three stage pipeline -…

Rancher - Creating a highly available container orchestration cluster on AWS

Note: In these examples I'm naming the project/org tastycidr, but this was a project done at work...aka not tastycidr.net Approximately a year ago, my organisation began development of a new platform based on a containerised microservices architecture. For us, it was uncharted territory - while some of…

Ansible Tower, Autoscaling and recycled IPs: a short treatise on cleaning up after yourself.

My organization's EC2 inventory is highly dynamic in nature. Our requirements vary considerably depending on the time of day, as well as, to a lesser degree, day of the week. As a result, we have a mixture of scaling policies based on schedules and based on performance or load metrics.…

MongoDB backups at scale - how one Ops guy got his "daily" backup job to run in under 24 hours.

There's more than one way to deal with dumping a lot of databases with slow tools. Awhile back, I wrote another article outlining how my organization handles cluster-wide backup and restore of our MongoDB clusters. The tool/methodology we use is great for quickly backing up, and more importantly, quickly…

Configuring a minimal LXD image for use with Ansible

I recently decided to spend some time tinkering with LXD. My organisation already uses Docker containers for certain services in production, but they do not address every use case. I wanted something lightweight that would behave more like a traditional instance for use in testing. Our automation tool of choice…

Backing up a MongoDB Cluster via EBS Volumes

Backing up and restoring a MongoDB Cluster via EBS Volumes Backing up and restoring MongoDB clusters at large scale is challenging. There are hosting providers like MongoDB Atlas which will handle it for you, but it can come with a price tag that many (like my organization) find prohibitive. This…