Measuring Code Integrity
I have been pondering this for quite some time now. How do we measure the 
code integrity. Code Coverage doesn’t really do it, and the number of test or test density also doesn’t.
I proposed to go back to the commercial essence of unit testing and bug slaying. In a team that doesn’t unit test, we have seen that 87% of the bugs found in the QA, are the result of sloppiness, a unit test would have found that bug, as the developer was aware of that case scenario, but just didn’t test it. Only 13% of the bugs found, where bugs in the design and scenarios that were not considered.
The thing is that a bug found in the QA must go back to the developers who must track it down and fix it, just to send it back to the QA (and hope that he didn’t add any other bugs). According to IEEE, this process cost $1000 per bug. While a bug found in development cost only $25-$100. That is 10-40 time less!
When checking this out, the direct benefit of coding with integrity is that the bugs don’t even reach the QA. If we could measure the bugs that are found in the development, we could know how effective our unit tests really are.
So if we measure the time it takes to fix a bug (Bug-Fix-Time) we should see that developers writing good unit tests will fix the bugs in less time then developers who don’t. We know that the industry standard is 6 days to fix a QA bug and minutes to hours to fix a failed unit test. That is a huge difference.
Lets measure the amount of unit test that failed on the developers desk/build server and the time it takes to fix them and know our BFT.
We should strive to get this down to a minimum to have a better code integrity.
3 Comments to “Measuring Code Integrity”
Recent Posts
- Top 5 questions to ask when checking references
- Unacceptable: Unit testing will take 20 years to catch on
- The 4 reasons why we DIDN’T choose Oslo
- Typemock Academy Launch
- The First Rule to Software Craftsmanship
Categories
- .NET Tests
- Agile
- Code Integrity
- Community
- Debugging
- Fun
- Management for Geeks
- Marketing
- Product
- Release
- Reviews
- SharePoint
- TDD
- Time Management
- Uncategorized
- Unit Tests
Archives
- August 2010
- May 2010
- April 2010
- March 2010
- February 2010
- December 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- December 2008
- November 2008
- August 2008
- July 2008
- May 2008
- April 2008
- February 2008
- January 2008
- December 2007
- November 2007
- October 2007
- September 2007
- August 2007
- July 2007
- June 2007
- May 2007
- April 2007
- March 2007
- February 2007
- January 2007
- December 2006
- November 2006
- October 2006
- September 2006
87% is a very high number – thanks for publishing! I’ve always wondered if there was a similar metric for static analysis tools (source code analysis). Static analysis tools may report a number of false positives or “don’t cares” and so organizations always want to measure whether they are investing the right time and energy into static analysis.
Hi Andy,
I am not talking about static analysis at all. I am talking about *logical* bugs, not *syntactic* ones.
Code Integrity is that the code DOES what it is supposed to DO. We need a suite of automated logical unit tests to make sure that our code is in integrity, and to manage this we need to measure our code integrity, we can do this by measuring our Bug Fix Time, our goal is to bring that number down, because the longer it takes to fix a bug, the less integrity our code has.
[...] I have debate this before (Measuring Code Integrity) and I think that measuring the Bug Fix Time is a direct measurement of the effectiveness of our unit tests. I have written a small application that sits on the developers desktop and we are starting to use it in Typemock. [...]