# Articles

• ### The Syntax Error That Killed Our Build for a Week.

December 12, 2015

Back in the day when I wrote in C and we used X11 with Motif, I was creating a new module based on an existing one, so I just copied and pasted the file from one xterm to another:

• ### Zero Downtime Migrations

August 15, 2015

For any sufficiently large application, you want to minimize interruptions to service while deploying new code. This is especially challenging for migrations on Heroku and other Platforms-as-a-Service, where you have a Catch-22 problem; you want to run your migrations, first, but you can’t run them until you’ve deployed the migrations to production. Usually, at this point, your new migration-dependent features are in the deploy, too. So when your application restarts, and the migration hasn’t run, yet, your poor application will trip some exceptions, and perhaps, create problems to your users. Of course, you can put the app into maintenance mode, but that creates more downtime for your users.

• ### Unintended consequences (and a disaster averted)

January 6, 2015

I want to tell you a scary-as-shit story, mostly so that my friends that have CCW permits might read it and not get killed by good intentions, because I love you all.

• ### Timecop time and database time

June 8, 2014

Timecop is a great addition to your testing toolbox, but if you’ve ever tried to use Timecop to interact with a database, and then got the most mysterious of existential errors:

June 1, 2014

Once a week, we run a query on our production system that takes a very long time to complete. Invariably, it times out because the default settings on Heroku are (quite reasonably) for web / API interactions; anything longer than 30 seconds is too long and the connection should be closed to let other requests through. How do you get around this, then? You could set up your database.yml file so that you connect to your production database from a developer workstation. This is actually quite useful, but opens up a hellacious security hole. Let’s not get started about the consequences of accidentally running rake db:drop:all! Moreover, you want your system to be automated as much as possible, and that means running your tasks on a Heroku instance. Fortunately, with a little hackery, you can modify the connection timeout settings within your application. Do this carefully, and make sure that it can not leak into your running web application dynos.

• ### When you need a rabbit, spec it

February 7, 2014

On my current project, we're using RabbitMQ. It's a bit of infrastructure that has to be present, and if it isn't, our integration tests will fail with mysterious error messages. We want our tests to be informative, so let's write a test that asserts that we have the requisite infrastructure in place.

• ### Speed up your integration tests with a jig

February 7, 2014

If you've written enough integration tests (with Capybara et al.), you must have noticed how much time your tests spend just logging into your web app. Even if it takes 1 second each time, it starts to add up. Here's a solution that I've written several times, now. I create a test "jig" that allows me to authenticate into my application with a single visit.

• ### Stop thinking like an employee

November 5, 2013

A few weeks ago, some of the managers got together for drinks after work. I started to reminisce about an old employer who had, what I thought, was an incredibly unique and reasonable way to handle the “vacation liability” problem. The old firm was a consulting firm, mostly military contracts, in the DC area. We were paid for every hour we billed, and we accrued vacation hours for every hour we were paid. It created an environment where the top performers could rack up so much vacation that they could not use it all. Many companies handle this with the familiar “use it or lose it” policy, which caps the accrued vacation hours to some limit. Instead, the firm let vacation accrue forever, with one modification: When you got a raise, your vacation hours were reduced by exactly the same percentage.

• ### Why Pivotal Labs

May 26, 2013

It is a time-honored tradition for Pivots to blog about their first few months at Pivotal. A typical day at Pivotal is strong work. It’s different from any previous job. It’s exhausting. After six weeks or so, however, the Pivots find their rhythm. I’m not going to write any more about that. I’ll include some links, below, so you can read them for yourself. I’m here to write about the two-years-later; When most developers are itching to move on to the next big thing. I’m still happily learning new stuff every day.

• ### Sencha Touch BDD - Part 5 - Controller Testing

May 18, 2013

Part 4 Introduced PhantomJS as an easy and faster alternative to headful Jasmine testing. Part 3 added jasmine-ajax so we can verify that stores and models react properly to back-end data. We also learned how to use stores to test views, without depending on a back-end server. In Part 2 I showed you how to unit test Sencha model classes in Jasmine. In Part 1 I showed you how to set up your Sencha Touch development environment to use the Jasmine JavaScript test framework.

• ### Sencha Touch BDD - Part 4 - PhantomJS

May 10, 2013

Part 3 added jasmine-ajax so we can verify that stores and models react properly to back-end data. We also learned how to use stores to test views, without depending on a back-end server. In Part 2 I showed you how to unit test Sencha model classes in Jasmine. In Part 1 I showed you how to set up your Sencha Touch development environment to use the Jasmine JavaScript test framework.

• ### Sencha Touch BDD - Part 3 - Testing Views and Mocking Stores

May 5, 2013

In
Part 1 I showed you how to set up your Sencha Touch development environment to use the
Jasmine JavaScript test framework. In
Part 2 I showed you to unit test Sencha model classes in Jasmine.

###I don’t normally test views, but when I do

There’s an old MVC mantra: “Fat models, skinny controllers and stupid views.” We don’t want complex logic in our views; that makes them hard to maintain. We don’t want any business logic in our controllers, either. That’s why we have models. Views should be so simple that they don’t require tests. There is a gray area with Sencha, however, where we found testing to be useful in our design process. It has to do with how views interact with stores.

Stores are essentially collections of models. They also encapsulate the persistence layer logic, separate from the business logic of the model. DataViews are a special class of Views in Sencha Touch that will consume a collection to generate a list-type view via a template.

Here are two goals for testing views and stores:

• Does the view consume the right fields from the store, without hitting the back-end for data?
• Does the store organize the data from the back-end, i.e. create the interface that is used by the view? Again, without hitting the back-end for data.

###Small steps and iterate

Let’s define a very simple view via tests, we’ll make it work using an in-line store, then we’ll refactor the store into its own class. Next, we’ll refactor the storage class to use a remote back-end.

Ext.require('SenchaBdd.view.MyView');

describe('SenchaBdd.view.MyView', function () {
it("has a list of colors", function () {
var view = Ext.create('SenchaBdd.view.MyView', {
renderTo: 'jasmine_content',
store:    {
fields: ['color'],
data:   [
{color: 'red'},
{color: 'green'},
{color: 'blue'}
]
}
});

expect(Ext.DomQuery.select('.favorite-color').map(function (el) {
return el.textContent
}).join(', ')).toEqual('red, green, blue');

});
});


Sencha does not come bundled with jQuery, so if you were expecting a DOM query like, $(“.favorite-color”), you might be surprised by the expectation. Ext has its own flavor of querying the DOM, using Ext.DomQuery.select. Try implementing the view on your own to make the test pass. It should look remarkably similar to this: Ext.define('SenchaBdd.view.MyView', { extend: 'Ext.dataview.DataView', xtype: 'myview', config: { itemTpl: '<div class="favorite-color">{color}</div>' } });  In your application, you probably won’t hardwire a store into the view. In fact, you will probably embed this little view inside a larger container, like so (this adds another tab to the sample app): --- a/app/view/Main.js +++ b/app/view/Main.js @@ -10,6 +10,11 @@ Ext.define('SenchaBdd.view.Main', { items: [ + { + title: 'Favorites', + iconCls: 'star', + xtype: 'myview', + store: 'mystore', + styleHtmlContent: true + }, { title: 'Welcome', iconCls: 'home',  Let’s create a store so our view will show us something: Ext.define('SenchaBdd.store.MyStore', { extend: 'Ext.data.Store', config: { storeId: 'mystore', fields: ['color'], data: [ {color: 'red'}, {color: 'green'}, {color: 'blue'} ] } });  In order to see this view, we’ll need to add it to our app.js file: --- a/app.js +++ b/app.js @@ -31,7 +31,11 @@ Ext.application({ ], views: [ - 'Main' + 'Main', 'MyView' ], + stores: [ + 'MyStore' + ], icon: {  Now, let’s go back and refactor our test so it uses MyStore instead of one that was hard-wired. --- a/spec/javascripts/view/MyViewSpec.js +++ b/spec/javascripts/view/MyViewSpec.js @@ -2,17 +2,16 @@ Ext.require('SenchaBdd.view.MyView'); describe('SenchaBdd.view.MyView', function () { it("has a list of colors", function () { + var store = Ext.create('SenchaBdd.store.MyStore', { + data: [ + {color: 'red'}, + {color: 'green'}, + {color: 'blue'} + ] + }); var view = Ext.create('SenchaBdd.view.MyView', { renderTo: 'jasmine_content', - store: { - fields: ['color'], - data: [ - {color: 'red'}, - {color: 'green'}, - {color: 'blue'} - ] - } - + store: store }); expect(Ext.DomQuery.select('.favorite-color').map(function (el) { return el.textContent  All of our tests should remain green, but we’ve removed the “fake” store from our test. ###Let’s get dynamic Static data stores are boring. The fun starts when you start talking to a back-end API. Let’s say that we have a server that responds to an end point of ‘/colors.json’ with a list of favorite colors. We can even “fake” it by placing a file in the appropriate place. Even so, we don’t want our tests to make network calls to the back-end. That’s not appropriate for unit testing. We’ll use Jasmine’s AJAX mocking helper, jasmine-ajax. At the time of this writing, the 2.0 branch had not been merged into the main line, and we need the 2.0 branch in order to work with Ext. cd spec/javascripts/helpers curl -O 'https://raw.github.com/pivotal/jasmine-ajax/2_0/lib/mock-ajax.js' git add ./mock-ajax.js  And we’ll create our first store spec in spec/javascripts/store/MyStoreSpec.js: describe('SenchaBdd.store.MyStore', function () { var store; beforeEach(function () { jasmine.Ajax.useMock(); clearAjaxRequests(); store = Ext.create('SenchaBdd.store.MyStore') }); it('calls out to the proper url', function () { store.load(); var request = mostRecentAjaxRequest(); expect(request.url).toEqual('/colors.json'); }); });  Notice that I call jasmine.Ajax.useMock() and clearAjaxRequests() in the set up block. This is because I want to wait until the very last moment to turn on ajax mocking. The Ext class loader might still be trying to load a class (via xhr), and the mocker will prevent that from happening. I also clear all previous requests (in case there were any left over, to prevent test polution). When you run Jasmine, you’ll get a test failure, “TypeError: Cannot read property ‘url’ of null” because we haven’t set up the proxy in the store, yet. Let’s do that. $ cat app/store/MyStore.js
Ext.define('SenchaBdd.store.MyStore', {
extend: 'Ext.data.Store',
config: {
storeId:  'mystore',
fields:   ['color'],
proxy:    {
type:       'ajax',
url:        '/colors.json'
}
}
});


Now, when you run the test suite, you’ll get a
different error! This is because the default settings for the ajax proxy enables caching, paging, etc. Things we don’t need, so we have to turn them off:

    proxy:   {
type:       'ajax',
url:        '/colors.json',
noCache:    false,
pageParam:  false,
startParam: false,
limitParam: false
}


Now our api test is green! You can even test this in the application by dropping a file, ‘colors.json’ into the public directory.

Let’s add one more test to finish things off.

  it('populates the collection', function () {
var mockedRequest = mostRecentAjaxRequest();

mockedRequest.response({
status:       200,
responseText: [
{color: 'red'},
{color: 'green'},
{color: 'blue'}
]
});

expect(store.getCount()).toEqual(3);
expect(store.getAt(0).get('color')).toEqual('red');
expect(store.getAt(1).get('color')).toEqual('green');
expect(store.getAt(2).get('color')).toEqual('blue');
});


Every mocked ajax request has a
response() method that you can use to inject your own response, synchronously, to your tests. This confirms that MyStore properly parses and arranges the received data so that it can be presented by the view.

We now have 2 tests that depend on a back-end to respond in a specified way. How do we keep these tests from drifting out of sync? Since we’ve mocked the store in our view test, we might never know if the back-end API changes!

You can wrap Jasmine expectation in functions so that they are reusable. Then we can mix this matcher into our view test to confirm that the store we use there is the same.

Let’s add this function to our
SpecHelper.js file:

function myStoreDataIsValid(store) {
expect(store.getCount()).toEqual(3);
expect(store.getAt(0).get('color')).toEqual('red');
expect(store.getAt(1).get('color')).toEqual('green');
expect(store.getAt(2).get('color')).toEqual('blue');
}


And we’ll replace our existing tests with a single line:

  myStoreDataIsValid(store);


We’ll also modify the MyViewSpec.js:

--- a/spec/javascripts/view/MyViewSpec.js
+++ b/spec/javascripts/view/MyViewSpec.js
@@ -9,6 +9,7 @@ describe('SenchaBdd.view.MyView', function () {
{color: 'blue'}
]
});
+    myStoreDataIsValid(store);
var view = Ext.create('SenchaBdd.view.MyView', {
renderTo: 'jasmine_content',
store:    store


Similarly, we can refactor our colors array into var:
diff
— a/spec/javascripts/helpers/SpecHelper.js
+++ b/spec/javascripts/helpers/SpecHelper.js
@@ -16,6 +16,15 @@ afterEach(function () {
domEl.setAttribute(‘style’, ‘display:none;’);
});

+var colorsJSON;
+beforeEach(function() {
+ colorsJSON = [
+ {color: ‘red’},
+ {color: ‘green’},
+ {color: ‘blue’}
+ ];
+});


And then use colorsJSON in our tests.

--- a/spec/javascripts/store/MyStoreSpec.js
+++ b/spec/javascripts/store/MyStoreSpec.js
@@ -18,11 +18,7 @@ describe('SenchaBdd.store.MyStore', function () {

request.response({
status:       200,
-      responseText: [
-        {color: 'red'},
-        {color: 'green'},
-        {color: 'blue'}
-      ]
+      responseText: colorsJSON
});
myStoreDataIsValid(store);
});


and

--- a/spec/javascripts/view/MyViewSpec.js
+++ b/spec/javascripts/view/MyViewSpec.js
@@ -3,12 +3,9 @@ Ext.require('SenchaBdd.view.MyView');
describe('SenchaBdd.view.MyView', function () {
it("has a list of colors", function () {
var store = Ext.create('SenchaBdd.store.MyStore', {
-      data:     [
-        {color: 'red'},
-        {color: 'green'},
-        {color: 'blue'}
-      ]
+      data:     colorsJSON
});
+    myStoreDataIsValid(store);
var view = Ext.create('SenchaBdd.view.MyView', {
renderTo: 'jasmine_content',
store:    store


### What we’ve managed, so far

The toy app is coming along. We’ve test driven a DataView that consumes a back-end API. We’ve added the mock-ajax library so we can unit tests our stores in isolation. We’ve even seen a few techniques for keeping our mocks from getting out of sync (although, if you’ve been paying attention, I’ve still left a gaping hole, that needs to be plugged).

• ### Sencha Touch BDD Part 2 -- Unit Testing Models

April 26, 2013

In Part 1 I showed you how to set up your Sencha Touch development environment to use the Jasmine JavaScript test framework. We’re going to take a bit of a breather from all the hard work we did last week. In this blog, I’m going to show you how to test simple models.

• ### Sencha Touch BDD Part 1

April 17, 2013

(tl;dr) A multi-part series of articles on how to test Sencha Touch applications. It uses Jasmine for unit testing and Siesta for integration testing.

• ### method_missing hazardous to your module?

February 22, 2013

We built an(other) object factory module for our current project and it looks a lot like all the others:

After a while, we noticed that the create methods were all exactly the same. Time for dynamic refactoring!

Unfortunately, this leads to rather bizarre error messages during testing:

Failures:



• ### It's The Volatility That Will Kill You

February 19, 2013

Volatility is what Pivotal Tracker uses to measure the consistency of your team’s work output. You can use that number to help you estimate the first approximation to answer the eternal question, “Will I make the deadline?”

• ### From customer requirements to releasable gem

May 12, 2012

One of the many pleasures of working at Pivotal Labs is that we are encouraged to release some of our work as open source. Often during the course of our engagements, we write code that might have wide-spread use. Due to the nature of our contracts, we can not unilaterally release such code. Those rights belong to the client. And rightly so. So, it is an even greater pleasure when one of our clients believes in "giving back" to the community, as well.

One such example is this modest gem, attribute_access_controllable which allows you to set read-only access at the attribute level, on a per-instance basis. For example, let's say that you have a model Person with an attribute birthday, which, for security purposes, cannot be changed once this attribute is set (except, perhaps, by an administrator with extraordinary privileges). Any future attempts to change this attribute will result in a validation error.

e.g.

May 12, 2012

# Deploy Strategies

If you look at the network graphs of heroku_san on github, you'll see a number of branches where the only change is the deletion of the following line from the deploy task:

• ### TDD Action Caching in Rails 3

March 28, 2012

On my current project, we needed to prove that an action cache was working as expected. Alas, the blogosphere had either out-of-date or unhelpful information. So, after many experiments, we came up with an RSpec test that does what we want. It seems ugly to me, and I hope there's a better way. The names have been changed to protect the guilty. Any resemblances to actual classes and methods are purely coincidental.

We needed to confirm that a certain action was cached. This action is preview in the brands controller. Using the usual Rails url helpers, we construct some fixture data.

Then we wrote our first test:

This won't work at all, however; because, in the test environment, caching is turned off.

So, we need an around block to temporarily turn caching on:

That's great, but the default cache store is the :null store, which, as its name implies, does nothing.

Better. But our tests still won't run because while ActionController uses the cache_store, Observers and Sweepers
use Rails.cache and that is only updated at boot time.

Did I mention that Rails.cache is an accessor for the global, constant, RAILS_CACHE. Ugh.

So, now, we can implement our method

But that is still not enough. caches_action
has an interesting performance enhancement; it doesn't actually set up the action caching unless caching is enabled at class load time. Since we're not turning caching on until test time, the caches_action method call in the controller class does nothing. We need to re-add it in our test spec.

This is ugly; it doesn't test very much (except the underlying caching module, and why bother testing the framework). At least it proves to ourselves that the action is cached and the cache key is what we expect.

Now that we've got caching under control, let's check cache expiration (using a Sweeper).

First, I create a cached object, in this case, just the string 'CACHED ACTION' and then I invoke the action, and then, I hope, the cache will be expired.

It doesn't really matter what happens in the #update method of the BrandsController as long as it updates a Brand object. A sweeper in Rails is a mix of Observer & controller filters, so all you need to do is "declare" it in the controller

Awesome sauce! Now our tests are red and I'm ready to implement the sweeper

And voilà! We have greenness.

So what have we learned from this? The Rails source is still your best friend when exploring a sticky problem. Caching is hard, and testing caching is even harder.

• ### Dry DevOps with HerokuSan

March 25, 2012

## Quiz time!

• ### DropBox + Git => Designer <3

April 5, 2011

One of the thornier problems in our workflow is knowing when assets are delivered from the designer and keeping them in sync with our application as they change. We used to use e-mail, Skype or sticky notes. The trouble is that the designer's file naming and directory structure were never quite the same as the application's /public/images directory, so direct comparisons were impossible and we ended with a lot bookkeeping to make sure that we didn't lose any changes. Our solution is to clone the project's git repository into a folder inside a shared Dropbox folder.

• ### When to do User Acceptance Testing?

November 18, 2010

"What does Scrum say about User Acceptance Testing? I am wondering if it should best be done within 24 hours of delivery, or at the end of a sprint..."

I'll dodge scrum, but answer how we do it here by framing this in terms of risk and velocity:

We prefer to get acceptance as soon as a story can be delivered to the acceptance/demo server. Intra-day is the best, but certainly within the same sprint when the work is done so as not to muddy velocity measures (a story isn't done until it is accepted, so there's incentive on all sides).

The longer a story sits out there (waiting to be accepted), the bigger the risk that it is "wrong" and needs re-work. Since the code base continues to evolve, fixes are harder the further they are away (in terms of commits) from the original change-set. This can slow down velocity. We want to reduce risk and maintain a steady velocity. Thus, the sooner a story is accepted by the stake holder, the better.

• ### The Elevator Speech

February 24, 2007

# The Elevator Speech

The purpose of the "speech" is to condense the project definition into something that would fit into the time it takes to ride an elevator. This is where all good ideas are pitched, apparently; you stalk an unwary VC until he or she is trapped inside the elevator has no choice but to listen to your prattling. If your idea has any merit, pray that the VC doesn't call on someone else to launch the project and leave you out in the cold.

• ### Using CVS for Web Server Farms

September 19, 2000

## Using CVS for Web Server Farms

by Ken Mayer (kmayer@worldpages.com)

• ### Crack Keys in Your Spare Time

September 19, 2000

or 'Hey buddy, got any spare CPU cycles?'

• ### A Modest Housing Proposal

September 19, 2000

I've been thinking about the next logical step in Silicon Valley human resource management. Already there are campuses where you never have to leave for (almost) all of your personal needs: the dry-cleaner will pick up and deliver; cafeterias, webvan.com and take-out-taxi/ meals on wheels keep you fed; health clubs, masseuse and the roving chiropractor keep your body functional; showers and locker rooms keep personal hygiene up to acceptable standards. So what's next in this reinvention of the company town?

• ### Go West, Young Man, Go West

August 16, 1999

## Maybe wax in your ears isn't such a bad thing after all

• ### The First Full Year of Wishful Thinking

December 24, 1998

It's hard to believe that it's been over a year since we left the house in Los Gatos and moved aboard the S/V Wishful Thinking. We made trips to Angel Island, South Beach Harbor, Richmond Marina, Pier 39, Vallejo YC, Golden Gate YC... covering something over 800 miles this year.

• ### The Smartest Boat in the Marina

November 10, 1998

Day 1: Moved in to my new digitally-maxed out Hunter 450 at last. Finally, I live on the Smartest Boat in the Marina. Everything's networked. The cable TV is connected to our phone, which is connected to my personal computer, which is connected to the shore-power lines, all the appliances and the security system. Everything runs off a universal remote with the friendliest interface I've ever used. Programming is a snap.

• ### A Travelogue of a Dive Vacation to the Bahamas, Hurricane and Shipwreck Included at No Extra Charge

November 6, 1998

• ### Fleet Week at Treasure Island

October 11, 1998

An eventful weekend, that's what we had. It started Friday evening about 6 PM as that was our window of opportunity to escape from the shallows of the marina entrance in the South bay. The winds had been blowing 25+ out of the north since early afternoon so it was going to be wet and splashy, but we really wanted to stay at Treasure Island Marina that night and on to Angel Island early the next morning. So, it was now or never. Wishful Thinking was actually the last of three to leave. Gypsy, a Union 38, left about 5 PM and Dragon Lady, a Cheoy Lee 40, left about 5:30. We gave them a call once we were underway. "Hope you guys are reefed" came their answer. "Reefed" I said, "hell we're motoring!" "We're making fine progress just quartering these seas a bit and heading into the 25+ knots".

• ### Treasure Island CyberCruise

June 1, 1998

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### Renaming Ceremony

April 26, 1998

• ### Projects List

April 23, 1998

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### Letter Home

April 10, 1998

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### Three Bridges

March 16, 1998

Berkeley Yacht Club, California, US

• ### The Birthday Month

February 27, 1998

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### On a Lighter Note

November 28, 1997

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### Musing on a Boating Life

November 27, 1997

Home port, Peninsula Marina, Redwood City, US
37N29.55' 122W15.98'

• ### Top 10 Tools for Lonely Sys Admins

October 15, 1995