Red Green Repeat Adventures of a Spec Driven Junkie

How I helped a founder to appreciate tests

This is an account of how I helped a founder to appreciate tests in the middle of his business pivot. I presented a way to test that solved the founder’s pain the most by showing how to test from the outside in.

Meet Ed

I met Ed from a co-working space we shared. We had lunch and he was telling me he was learning to code Rails at night school so he could build out the site for his business idea.

I really admire Ed to spend the time and energy to learn how to build his vision. I have worked with too many people that just ‘focused on the business’ and have zero understanding of the technology (or how it can complicated!) I’m sure Ed understood how important testing was since he managed testing in his previous career.

Don’t forget to test…

So, I was reminding him all about tests and how it’s really important to get setup early. (At the time, I just switched jobs and the new Rails app had very good test coverage which was a dream.) I didn’t want Ed to have the same nightmares I did at my previous job with no tests in the app.

Ed plainly told me: “I know tests are important, but I feel it’s more important to get a product to market fast, then worry about tests later.”

A few months later…

When I had lunch with Ed again, he immediately asked: “How can I get tests going in my Rails app??”

I took this opportunity to not just tell him: “I told you so!” I wanted him to be really on board with tests, specifically specs.

Reviewing Ed’s app

So, I asked for repo access to look at what is going on with his Rails app and demonstrate first hand how specs can work for him.

Ed gave me repo access that afternoon and I was impressed. Ed’s app was not anything I was expecting a Rails app to be… The site wasn’t just a Rails app… It is a Shopify app too!

Getting Ed’s site to boot took me a little while since I’m not familiar with Shopify. There was a lot built up because Ed worked on it, contractors worked on it, and recently a new developer joined too.

I honestly wanted to get Ed’s app setup with rspec, write a few specs and let Ed’s team do the rest with inside out tests.

Gut feeling…

Even getting specs going would not benefit Ed that much and probably deter him from tests even more. The best time to get specs going was during the first lunch!

Also, I knew the business was changing as well so getting Ed’s Rails app under specs might not be super beneficial. Specs will be scrapped later when business requirements change (which they will.)

Outside in tests?

It’s not specs, but for Ed’s situation, outside in tests might be a really good solution with everyone working on it and business variability.

Pros for going outside in

Outside in tests would put the code base under some automated testing but still allow a lot of flexibility to change it:

  • automated because the test script can run against it repeatedly
  • flexible because the script only tests very high level features (i.e login). Allowing lower level details to change (i.e user authentication method).

Cons for going outside in

The drawback is tests take longer to develop upfront and run slower. The test script is tightly coupled to the site. Any small changes to the front part of the site will require changes in the script.

Preparing talk

With this approach, I prepared a talk for Ed to explain how to test from the outside in using Selenium. I wrote two Selenium scripts, one for each user type of his platform, to go through the user role.

Presenting outside in testing

When Ed saw the Selenium script going through each step of his site: login, creating an order, order verification, order shipment verification, receipt etc. I can see many neurons firing. It was exactly what he was looking for in testing:

  • Quick to integrate to the site: the selenium script I wrote was working on his site right away. No need to reconfigure his site to make it work.
  • Easy to start tests: no need to change up existing code inside the site to support tests.
  • Visual: watching the browser going through the motions is very satisfying. There is no doubt what is happening and being tested. It was just like having a client use the site.

Best part of outside in testing

The script covers testing the really mundane parts of the site, making sure old features worked when new features are added.

I set Ed up with the script I wrote. He was all over it and got it going right away.

Since I just wrote it in plain Ruby, Ed and his new developer picked it up easily and ran with it.

The real pain of not testing

After I presented Ed the talk, he told me exactly his pain: He demoed his site to potential clients and every feature he worked on directly was fine. But, as soon as the client started playing with other parts of the site, previous features that worked (but he didn’t test recently), broke.

Ed realized having tests for critical paths of his site would make his life better.

Another testing success.

Epilogue

I saw Ed later for coffee and said they took my script and set up integration tests for his site. Once he saw there was more than one way to test, it made it easy to invest time in setting it up.

Lesson I learned

Getting founders on board with testing can be a tricky thing. By taking the right approach by showing how automated tests can perform slow and mundane work appeals to any founder with a tight budget.

At the same time: starting with tests is ideal… But this tweet from Patrick Mackenzie got me:

I definitely agree with, so there definitely needs to be a balance!

Engineering cannot just be testing the site, there needs to be features. Business needs to have some certainty with the tech features to close the sale.

Balance both out by having lots of tools to test and develop. Outside in, inside out, generative testing, and more!

Go forth and test (as needed!)