How I like to test

tl;dr only test the code you wrote for behavior; no more, no less.

I was having lunch today with an iOS developer. He is currently working in a Ruby on Rails shop… He’s the only iOS developer there… but there are 14 Ruby on Rails developers! He complained the rails guys tested everything. That’s cool with me. It’s a good habit to keep.

Our test suite is taking almost 25min on a single machine (something I’m not happy about…) but, it’s solid coverage and keeps us confident in our application. Then my friend told me how long it took them to run their tests… the team had to: spin up 20 instances of Circle so the tests would complete in 25minutes.

Wow, 20 instances to get 25min… I have read and heard people complain if 6000 tests ran over 1 minute on their dev machine. My 25min felt sluggish… but 20 instances to get 25min?! That’s inssane.

That triggered a thought to me: they were testing too much and probably in the wrong way. Tests should be light and fast, especially unit tests. Test coverage should be just enough for the “happy path”. Tests should be there for any ‘bugs’. Tests should only be heavy for integration tests.

How I test?

These are the things I like to test for:

• Test for behavior: this I learned from Sandi Metz’s awesome book: Practical Object Oriented Design in Ruby. Basically, it asks:

  • Does the object do what I think it does?
  • Does it handle the basic edge cases? (missing arguments, null/nil references, incorrect arguments.)
  • When given X input, does the expected thing come out? Test that sending X input in, Y comes out.

• Tests can be made lighter by involving the database less. Making an actual save on the database slows down tests, only because the database makes extra effort to ensure data is written to disk. In rails, testing if an object is .valid? is easily 10x faster than .save. (I’m sure my current project is guilty of this.)

• Test the happy path, the expected behavior of your function(s). Then test the converse, the unhappy path. Like the positive and negative. When is it expected to work? When is it expected to fail?

• Any ‘bugs’ encountered after the feature has been deployed should have a test written just for it. Why? Because the original set of tests did not find it,

Things I do not test

Well, this is a little tougher topic, but these are the guidelines I follow.

• Private methods: I don’t test any private methods in my code. I want to keep my code light enough so I can refactor it. Since I’m testing for behavior, that’s most important. How I achieve that behavior is another question and its solution will change, especially with refactoring!

• External libraries: I avoid testing any external library I import. I expect the library to be tested and if I’m not confident, I’ll do some light testing against the library for the happy path (or any ‘hacked way’ I used the library). But there’s a reason why it’s a library and it’s not code I wrote.

• Every possibility: I don’t write tests to test every possibility… there’s permutation testing and that is much better to do the testing instead of writing out the test myself. (I haven’t experimented with this yet.)

• View testing: I avoid this mainly because I treat the browser like a library. If there’s anything expected on the browser side, I test the javascript I have written there.

The things I don’t test are things I did not write. I want tests to keep me honest about MY code, not someone else’s. At the same time, I want the tests to support me, not chain me to a certain implementation.