Posts Tagged - testing

On Flaky Tests, Time Precision, and Order Dependence

A flaky test is one that fails unpredictably, without a corresponding change in the code under test. These often show up in CI runs where a test unrelated to any change made suddenly fails, and then mysteriously passes when re-run. There are many type and causes of “flakes”. Today I want to talk about flaky tests caused by time precision, and time-order dependence.

The specific tools mentioned here will be Ruby, Rails, and PostgreSQL specific. But analogs exist in nearly all other languages and frameworks, so the techniques are broadly applicable.

Frye meme: Not sure if I broke something, or it's a flaky test
Not sure if I broke something, or it's a flaky test…

Read on →

Sensible Testing is About Sensible Design

If you’ve not yet seen Justin Leitgeb’s GoGaRuCo talk, Sensible Testing, go check out the slides. I hope the actual talk is posted soon; I’m sure it’s more rich and full of context.

Overall I agree with most of Justin’s points. However, I get the feeling that he, like many folks, approaches testing as a method for verifying the correctness of code. As I understand the thesis, CUPID is to testing what SOLID is to design.

While I’m not opposed to laying down some names and concepts to improve the state of testing, I don’t believe testing should be the goal. It’s not about testing. I believe testing is a tool to be used in guiding your design.

Read on →

From Testing to Test-First to Test-Driven

When I started writing tests, around 2005, I was stoked just to have the tests.

When I started writing tests first, around 2006, I was excited because I was Doing The TDD.

A couple of years later I found that writing tests was getting really painful. Painful because they were so damn hard to set up, and painful because I had to wait too damn long (on the order of 10-15 minutes) for the test suite to run. I reacted to the pain by changing how I write my tests; I discovered mock objects. My tests got faster, but they were still painful.

A realization

In 2008 I was talking with Corey Haines about test pain, object-oriented design, and “listening” to the former to influence the latter. Scott Bellware also contributed much insight, forcing me to really think about what I hoped to gain from writing tests.

Those conversations help to crystallize it for me: the root cause of the pain was not the tests, but the design of the code under test. I had been doing test-first development, not test-driven design.

In the years since I’ve honed my technique for driving design by listening to tests and I continue to seek out the ideas and experiences of others.

The Ruby and Rails communities have accelerated this path for many. I would say it’s not uncommon for new folks to get started where I was in 2005 or 2006. What’s more exciting is the growing numbers who are starting to feel some pain in how they test. The next step is to become more aware of that pain; lower your pain threshold, and then make it stop hurting.

It’s a journey

It was by no means an overnight endeavour. It literally took years of work for me to figure this out, and I’m I’m still learning. I hope by putting my experience out there, yours can be better, faster, MOAR!

Read on →

Prefer Dependency Injection to Service Location

There is currently a thread running over in the StructureMap Users mailing list asking if we really need constructor injection when using an Inversion of Control container. Before any one rips off on a rant let me say that I worked with Jon in my former life and I’m fairly certain he’s merely conducting a thought experiment, trying to sure up his own beliefs. A worthwhile exercise, if you ask me.

At any rate, I have a few points I wanted to throw out there; most of them basic and mere reiterations of the words of others, but I’m gong to do it anyhow!

Read on →