I have been pondering this for quite some time now. How do we measure the 
code integrity. Code Coverage doesn’t really do it, and the number of test or test density also doesn’t.
I proposed to go back to the commercial essence of unit testing and bug slaying. In a team that doesn’t unit test, we have seen that 87% of the bugs found in the QA, are the result of sloppiness, a unit test would have found that bug, as the developer was aware of that case scenario, but just didn’t test it. Only 13% of the bugs found, where bugs in the design and scenarios that were not considered.
The thing is that a bug found in the QA must go back to the developers who must track it down and fix it, just to send it back to the QA (and hope that he didn’t add any other bugs). According to IEEE, this process cost $1000 per bug. While a bug found in development cost only $25-$100. That is 10-40 time less!
When checking this out, the direct benefit of coding with integrity is that the bugs don’t even reach the QA. If we could measure the bugs that are found in the development, we could know how effective our unit tests really are.
So if we measure the time it takes to fix a bug (Bug-Fix-Time) we should see that developers writing good unit tests will fix the bugs in less time then developers who don’t. We know that the industry standard is 6 days to fix a QA bug and minutes to hours to fix a failed unit test. That is a huge difference.
Lets measure the amount of unit test that failed on the developers desk/build server and the time it takes to fix them and know our BFT.
We should strive to get this down to a minimum to have a better code integrity.
87% is a very high number – thanks for publishing! I’ve always wondered if there was a similar metric for static analysis tools (source code analysis). Static analysis tools may report a number of false positives or “don’t cares” and so organizations always want to measure whether they are investing the right time and energy into static analysis.
Hi Andy,
I am not talking about static analysis at all. I am talking about *logical* bugs, not *syntactic* ones.
Code Integrity is that the code DOES what it is supposed to DO. We need a suite of automated logical unit tests to make sure that our code is in integrity, and to manage this we need to measure our code integrity, we can do this by measuring our Bug Fix Time, our goal is to bring that number down, because the longer it takes to fix a bug, the less integrity our code has.