Extending Test Frameworks

Eli Lopian    .NET Tests, TDD    

One really nice feature of TypeMock.NET 4.0 is the ability to extend any/all testing frameworks.

Each test framework has (or lacks) its own extension mechanism, making it hard add extension to all the frameworks. With TypeMock we can now write an new test decorator that will work for all frameworks, including vsTest and nunit, and all runner-tools including TestDriven.NET and ReSharper.

Actually, It is possible to decorate and extend ANY method! As long as it is run with TypeMock enabled. So it is now really simple to implement Roys Enterprise Services to do database testing, and Ian’s Even simpler way for ALL testing frameworks, without rewriting the test framework, extending base classes or forcing ALL tests in fixture to rollback.

We will be able to write the following test:

[Test,Rollback]
public void Insert()
{
      //Do some inserts into the DB here and automatically roll back
}

Here is the code that does it
[UPDATE: Fixed according to Jonas comment]

[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method,
  AllowMultiple = false, Inherited = true)]
public class RollbackAttribute : DecoratorAttribute
{
    public override object Execute()
    {
        using( new TransactionScope(TransactionScopeOption.RequiresNew))
        {
            // run the test 
            return base.CallDecoratedMethod();
        }
    }
}

It is possible to rollback on all tests by decorating the class

[TestFixture,Rollback]
public class DatabaseTests
{
      //Do many DB tests here and automatically roll back
}

We have implemented the [VerifyMocks] and [ClearMocks] using this technique.

Natural Mocks becoming more intelligent

Eli Lopian    Product, TDD    

Natural Mocks automatically mocks everything. This includes return values. So the following code:

using (RecordedExpecations recorder = RecorderManager.StartRecording())
{
   DummyClass.DoWork();
   recorder.Return(new SqlInt32(1));
}

Won’t work as expected because SqlInt32 is being mocked.

This has bugged me a few times and I simply created the return value outside the recorder.

SqlInt32 mockValue = new SqlInt32(1));
using (RecordedExpecations recorder = RecorderManager.StartRecording())
{
   DummyClass.DoWork();
   recorder.Return(mockValue);
}

But this is not really nice, and a few customers have pointed out that this could be annoying!

So we are going to implement a feature that will know that you are creating a return value and will not mock it!
This way the first code will work as expected. Of course if you want to return a mocked value you will have to create the mock before, this might lead to non backward compatibility.

To those who want the first code to work as before (although I don’t think that anyone would), mock the return value before calling the recorded statements, and change the above code to:

using (RecordedExpecations recorder = RecorderManager.StartRecording())
{
   SqlInt32 mock = new SqlInt32(1);
   DummyClass.DoWork();
   recorder.Return(mock);
}

Of course if you are going to do that, you will have to define the behavior of the SqlInt32 and you might as well use ChainedMocks

using (RecordedExpecations recorder = RecorderManager.StartRecording())
{
   DummyClass.DoWork().Value;
   recorder.Return(1);
}

What do you think about this feature?

Design and Testability – YAGNI

Eli Lopian    .NET Tests, TDD    

After reading Roy’s Post and Oren’s Post who strongly side on Designing for testability I suddenly felt a De Ja Vu.

De Ja Vu

A few years ago, I was managing a project in a big company. One of the features that we where developing was a caching system. We needed the caching to boost performance on a multi location, multi server application.

We developed in a normal Waterfall methodology, and the architect (a really funny and talented guy) came up after a few weeks with his design.
The design was really well thought out and the cache was ready to support – multiple databases with many different kinds of databases along with other things. At the time we believed that we had to design the system with the future in mind.

The question did arise:
Do we need to support Multiple Database Instances?

The Answer was, No, Not really, but if we are going to support it later then the redesign will take us ages.

When I asked the team to read and tell me what they think about: “You Ain’t Gonna Need It“, everyone took this personally. The architect started to feel that his job has been taken away from him:
“I am getting paid to think about future problems” and the developers where also taken aback. They have all been trained to create flexible systems and not flexible processes.

I failed to convince them, and a couple of years later, the caching system became what I call “a black hole”. No one knew exactly how it works, and everyone was scared to change it, because it was soo complex. The developers now said: “We didn’t really need multiple database instance, we spent ages designing and developing this feature, and every bug we find, has to support this feature too, which just complicates matters, because no one uses it but we cannot remove it.”

Still when I talk to them about YAGNI, they all say: “But when we thought ahead and designing the <fill in whatever feature>, we saved weeks and could implement <another featrure> in days. It would have taken us weeks without it”.
I have to remind them that it took months to design, implement and test before we even needed it.

YAGNI, Low Coupling and Testability

So, what has this got to do with Oren and Roy’s posts.

Well, I am getting the same feeling from them that I got from my architect.
Oren, so you found that a “Couple of weeks after introducing the IFileWriterFactory” you found another use for it, and it saved you time.

Good for you!
But this is NOT YAGNI, the opposite. It sounds just like that <fill in whatever feature> that my team was so proud that it had managed to predict.
What about all the other features that have no use? The price of doing this everywhere can be quite expensive. Most of the time (apart from tests) – You Just Ain’t Gonna Need It.

There was a time when we had no choice, you had to either Design for Testability or not test at all. But in these times this is no longer true.

So, Does testing - driving your design, actually create the best design? I am not sure.
It does one thing for sure, It creates a low coupled (modular) design. But low coupling is NOT a silver bullet. A good design has to take and BALANCE many factors, this is a complex task. Adding Testability to these factors might unbalance your design.

Quiz: Which is easier to maintain:

File.WriteAllText(filename, text);

or

using(TextWriter writer = IoC.Resolve<IFileWriterFactory>().Create(filename))
  writer.Write(text);

Multiple Asserts in a Single Unit Test

Eli Lopian    .NET Tests, TDD    

There is a debate going on about having one assertion per test.

James Avery argues that what he’d really like to see in a unit test framework is the ability to run (and fail) multiple asserts within the same test and Roy Osherov still argues that there is value in having one assertion only.

I am less inclined to force a one assertion per test as it might add unneeded burden on the developer, who has to set up the same test scenario for each test. This will require Refactoring the test and Extracting a setup method. On the other hand I would not want the tests to continue to run once the first test has failed, as Roy correctly pointed out, we are in a dirty state.

As we know, there is no Silver Bullet we are going to have to rely on the developers to take the call and decide what is best for them.

I am in favor developers being guided by:
The Test Should Test Only ONE scenario

This does not mean one assert! it means one scenario.

If we need to assert several items to verify that the scenario works, we should do it in the same test (This is how all mocking frameworks work anyway).

In the examples that James brings, each test verifies one scenario but requires several asserts. I believe that this is good practice.

14 facets of TypeMock.NET and Designing for Testability

eli    .NET Tests, TDD    

There has been much talk about Designing for Testability lately.
Basically the argument is:

Here is what 14 of are our community have to say about it:

Still Debating