Measuring Effective Unit Tests

What is an effective unit test?
A good unit test (via Jeremy Miller) is

  • Atomic
  • Independent & Isolated
  • Intention Revealing
  • Easy to Setup
  • Runs Fast

But how do we know that the test is effective?
The biggest value of unit tests come when they fail. When a test fails, and we fix it, we have saved considerable time in discovering the bug, pinpointing the failing scenario and fixing it. The more bugs we find early on and the faster we fix them, saves us headaches and time. We also know that the longer we leave a bug in the system, the harder it is to fix.

I have debate this before (Measuring Code Integrity) and I think that measuring the Bug Fix Time is a direct measurement of the effectiveness of our unit tests. I have written a small application that sits on the developers desktop and we are starting to use it in Typemock.

image

This is what the small application looks like. Each developer can see how long it takes them to fix a test. We still have to tweak the calculation.
We are thinking of counting the failure time only after the first test is run (for TDD teams) although this will not count known bugs, as it is standard to write a test before solving them.
We do consider a group of test that fail together in one run and are solved together in another run as one test.

6 comments
  1. I was once asked about ‘the effort’ for completing test coverage and how much it costs the company writing all the tests, I summarized my response in this blog post and I think it’s relative to what you’ve written here.

    http://vsteamsystemcentral.com/cs21/blogs/brian_johnston/archive/2009/04/07/costs-of-unit-testing-vs-not-unit-testing-and-the-last-15-for-code-coverage.aspx

    I would *love* get a copy of this program your describing if you were to ever to share it with the rest of the world.

  2. It’s hard to find your articles in google. I found it on 18 spot,
    you should build quality backlinks , it will help you to rank to google
    top 10. I know how to help you, just type in google – k2 seo tips

  3. I read a lot of interesting posts here. Probably you spend
    a lot of time writing, i know how to save you a lot of work,
    there is an online tool that creates high quality, SEO friendly articles in seconds, just type in google – laranitas free content
    source

Add Comment

Required fields are marked *. Your email address will not be published.