# Tag: devops

• ### Our CTO Sleeps Well At Night

July 30, 2018

I am PacerPro’s chief technical officer, which means that I get the first phone call at 3 am when things go pear-shaped. Fortunately, I don’t get many of those calls: I’ve built a development team and process that delivers reliability as a core feature.

• ### On Call Duty

March 11, 2018

Recently, there was a flurry of tweets about on-call and SRE duty schedules. It triggered a memory of a conversation at a manager meeting about a new product offering:

• ### Do Not Push to Red Builds

July 8, 2017

There's a pattern to disasters: When you examine the post mortems, its not just one failure, but a cascade of multiple failures that overwhelm the system's safety features. Pushing to a red build risks piling one failure upon another.

• ### Memory Leak Whack-a-Mole

October 27, 2016

Memory leaks happen. They shouldn’t, but you just can’t squash enough of them. Or the leak is so slow that it takes hours to manifest. The trouble with memory leaks is that they are permanent for the life of a process. Once a process has expanded its VSZ, there’s no way in Unix to shrink it. So your process starts to bloat, eventually your servers page and thrash. Performance tanks. Then your pager starts going off and there goes your beauty sleep.

• ### When You Need a Rabbit, Spec It

February 7, 2014

On my current project, we're using RabbitMQ. It's a bit of infrastructure that has to be present, and if it isn't, our integration tests will fail with mysterious error messages. We want our tests to be informative, so let's write a test that asserts that we have the requisite infrastructure in place.

• ### Dry DevOps with HerokuSan

March 25, 2012
1. How many times (each day) have you typed this at your console?
git push heroku master and then forgotten to run
heroku run rake db:migrate --app yellow-snow-3141 or
heroku ps:restart
2. Does your script support a multi-stage environments?
3. Do you remember how to get to the application's console process?
4. Is your application's configuration consistent across all stages?
5. Are you deploy scripts tested?

• ### Using CVS for Web Server Farms

September 19, 2000

Let's assume that you've adopted the techniques described in Using CVS for Web Development (or [Dreilinger]) and you are the Release Master. Now imagine that instead of one web server you have a web server farm of 10 or 100. How do you prevent chaos and carpal tunnel syndrome from keeping you chained at your desk for days on end? Does this look familiar:

devops - Ken Mayer