What is an effective unit test?
A good unit test (via Jeremy Miller) is
- Atomic
- Independent & Isolated
- Intention Revealing
- Easy to Setup
- Runs Fast
But how do we know that the test is effective?
The biggest value of unit tests come when they fail. When a test fails, and we fix it, we have saved considerable time in discovering the bug, pinpointing the failing scenario and fixing it. The more bugs we find early on and the faster we fix them, saves us headaches and time. We also know that the longer we leave a bug in the system, the harder it is to fix.
I have debate this before (Measuring Code Integrity) and I think that measuring the Bug Fix Time is a direct measurement of the effectiveness of our unit tests. I have written a small application that sits on the developers desktop and we are starting to use it in Typemock.
This is what the small application looks like. Each developer can see how long it takes them to fix a test. We still have to tweak the calculation.
We are thinking of counting the failure time only after the first test is run (for TDD teams) although this will not count known bugs, as it is standard to write a test before solving them.
We do consider a group of test that fail together in one run and are solved together in another run as one test.
[REMOVED]
Great post Brian,
I will send you a copy of the application, although it is very early stage. We are trying to see how this metric helps.
I was once asked about ‘the effort’ for completing test coverage and how much it costs the company writing all the tests, I summarized my response in this blog post and I think it’s relative to what you’ve written here.
http://vsteamsystemcentral.com/cs21/blogs/brian_johnston/archive/2009/04/07/costs-of-unit-testing-vs-not-unit-testing-and-the-last-15-for-code-coverage.aspx
I would *love* get a copy of this program your describing if you were to ever to share it with the rest of the world.
It’s hard to find your articles in google. I found it on 18 spot,
you should build quality backlinks , it will help you to rank to google
top 10. I know how to help you, just type in google – k2 seo tips
I read a lot of interesting posts here. Probably you spend
a lot of time writing, i know how to save you a lot of work,
there is an online tool that creates high quality, SEO friendly articles in seconds, just type in google – laranitas free content
source