The Humble Dialog (and famous musketeers)

Eli Lopian    .NET Tests    

I have read Jeremy’s Build you own CAB series (Build your own CAB Part #2 – The Humble Dialog Box). Jeremy is building the series with the help of  d’Artagnan the famous Musketeer wanna-be.

Due to both his pride and an urge to ingratiate himself with those he wished to join, d’Artagnan is challenged to a duel by three musketeers Athos, Porthos, and Aramis. d’Artagnan had a little surprise for his opponents. He did not divide his concerns, instead he embraced them, and found himself fighting all 3 together. The 3 started fighting with each other over who will fight d’Artagnan first, but they all ended up fighting Cardinal Richelieu’s guards, who threaten to arrest them because duels are forbidden by royal decree…

I am going to show you just how we can manage to unit test a Simple Dialog without separating concerns and overcome the forbidden decree that GUI is hard to test.

I will use Jeremy’s requirement:

If the user attempts to close the XYZ screen without saving any outstanding changes, a message box is displayed to warn the user that they are discarding changes.  If the user wishes to retain the outstanding work, do not close the screen.  The message box should not be shown if the data has not been changed.

Unlike Jeremy, MessageBox will be no problem at all to test, actually the tests are quite straight forward and will give you the developer the freedom to choose (and refactor) the code when needed.

Here are the tests:

[Test]
[VerifyMocks]
public void CloseTheScreenWhenTheScreenIsNotDirty()

Cool Ah!, we mocked the unmockable MessageBox, we also mocked the SimpleForm.isDirty, we actually tested the logic that we needed, this makes the GUI very testable with no effort at all.

The mocking is done in the using block where all calls are mocked and the behavior is set. Verification that the expectations where called is done using the [VerifyMocks] attribute. We could also test the message that is being displayed on the message box, but that is another test.

You should of course separate the GUI class when it grows bigger, but your test will still pass even if you do separate the concerns! Unless you change the message box to one of your own, but you have the ability to really do Just In Time Design (YAGNI)

Just one more tip for those of you who will try this technique. It will be a good idea to mock MessageBox.Show in all cases, just in case the test fails-> we don?t want the tests to hang with the message box. Just use .RepeatAll or FailIfCalled to make sure that we don’t fail if the MessageBox is not shown.

Released TypeMock 4.0 Beta

Eli Lopian    .NET Tests    

We have released a beta version of TypeMock 4.0, this should be a really stable version. We intend to release the final version within 2 weeks.

There is a lot of new stuff in this version. you can read about it in the Release Notes.

Some nice stuff are:

The new TypeMock Decorators.

Using these decorators will work for All test frameworks and will automatically verify mocks

[Test]
[VerifyMocks(Timeout=150)] // mocks are always verified
public void TestVerificationWithTimeout()
{
   // setup mocks

   // run tests

   // optional assert
   Assert.That():
}

Generic Reflective API

Use generic to create reflective mocks.

[Test]
[VerifyMocks] // mocks are always verified
public void Test()
{
   Mock mockControl = MockManager.Mock<ClassToMock>();

   new ClassToMock(); // This is mocked
}

Intelligent Natural Mocks

When creating a mocked return value, within a recording block, TypeMock.NET will not mock the return value

using (RecordExpectations recorder = RecorderManager.StartRecording())
{
  MockedType.CallAMethod();
  recorder.Return(new SomeReturnType(1,2));// SomeReturnType is NOT mocked
}

TypeMock Case Study in Corillian

Eli Lopian    .NET Tests    

We have completed a case study of implementing TypeMock in Corillian. Corillian is the company that creates Online Banking Software, and you can imagine how important security and quality is.

Corillian is the company that Scott Hanselman is the Chief Architect. Scott has promised me that he will write an article about TypeMock as well.

We found that the biggest value of using TypeMock is that it eased the adoption of unit testing to all the developers. One of the main obstacles in creating Agile Development Teams. The team members where really happy to adopt testing early, and had both the freedom to develop in their own style without compromising the testability of their code.

The value of having agile teams is well know and IBM pitches the efficiency of agile programming. But we have managed to find some interesting metrics:

• The number of tests in one team grew from 100 to 600 within 1.5 months! That’s a 500% increase.
• The test coverage (amount of code covered by the tests) grew by 50% within 2 months
• The actual tests are 25% faster to code! that saves time to create more features.

You can grab the Case Study pdf file off the TypeMock site

How did we do that?!!

Eli Lopian    .NET Tests    

If you have read my previous post about the TypeMock Test Attributes you probably have noticed that the attributes are test framework agnostic!

So how did we do that?

We tried to add our attributes using implementing the hooks of each framework. But we found this way to be unstable:

• mbUnit is the easiest framework to extend, it is build with extensions in mind, but the extensions are version specific
• nUnit has just started supporting extensions and the system is fragile, the extension system requires deploying to each nunit installation (and TestDriven.NET too) and needs to be redistributed for each nunit version as the extensions are version specific.
• msTest (the Visual Studio Team System) currently has no means to extend and add attributes.

Then we remembered that TypeMock has an AOP framework. We can inject the Attributes! This is what we did.

We spiked and wrote the following code

[TestFixture]
public class Spike
{
    [SetUp]
    public void InjectVerifyMocks()
    {
        foreach (MethodInfo info in this.GetType().GetMethods())
        {
            if (info.GetCustomAttributes(typeof(VerifyMeAttribute),
                false).Length > 0)
            {
                Mock m = MockManager.MockAll(this.GetType());
                MethodInfo keepInfo = info;
                m.AlwaysReturn(info.Name, new DynamicReturnValue(
                    delegate(object[] p, object c)
                    {
                       keepInfo.Invoke(c, p);
                       MockManager.Verify();
                       return null; // void method
                    }));
            }
        }
    }
    [Test]
    [VerifyMe]
    public void SpikeTest()
    {
        Mock m = MockManager.Mock<MockedClass>();
        m.ExpectAndReturn("Method",10);
    }
}

Ok, so what are we doing here?

In the setup we are scanning all the methods for a custom attribute. When we find such a method we mock ourselves!!! and change the behavior of our methods to return a dynamic return.

The dynamic return will call ourselves again!!! (via reflection) and then verify the mocks.

When we run the test (in all frameworks) it fails with VerifyException.

TestCase 'Spike.SpikeTest'
failed: TypeMock.VerifyException :
TypeMock Verification: Method MockedClass.Method() has 1 more expected calls

This works because while we are in the DynamicReturnValue Delegate, TypeMock is turned off and no methods are mocked. So we don’t recursively call ourselves, but on the other hand this means that our mocks are all turned off. 

This was just a spike to prove that it was possible, it took more time to implemented the extension system that will work for all testing frameworks.

I will publish how to extend and create your own attributes when we release the beta version.

New Best Practice Pattern

Eli Lopian    .NET Tests    

In our next release TypeMock.NET will enhance the Best Practices to use Mocks and we have added 2 new ways to use mocks

• Scoping the Mocks
• Using Test Attributes

Scoping the Mocks

In this way the mocks are scoped using the following pattern

using (new MockScope()) // can set an optional timeout
{
    // setup mocks

    // run tests

    // optional assert
    Assert.That();
}
// mocks will be verified

The advantages of using this method are:

• No dangling mocks will be left between tests.
• All Expectations will be verified.

The disadvantage is:

• Unverified mocks will take precedence and hide the Assert Failure.

Using Test Attributes*

We have added an even better way to use mocks, by defining TypeMock custom attributes to decorate the test methods.

[Test]
[VerifyMocks(Timeout=150)] // mocks are always verified
public void TestVerificationWithTimeout()
{
   // setup mocks

   // run tests

   // optional assert
   Assert.That():
}

Once we put the [VerifyMocks] attribute on the test, the test will automatically verify all mocks.

The advantages of using this method are:

• No dangling mocks will be left between tests.
• All Expectations will be verified.
• Asserts will take precedence over mock verification.
• Mock will be cleared in all cases (even if a failure was thrown)

It is possible to use [VerifyMocks(Timeout=100)] that will fail the tests if not all expectations where met within the timeout.

There will also be a [ClearMocks] attribute that will always clear the mocks between tests (without verification)

These attributes can be used on the test class level too so the following will verify mocks after each test

[TestFixture]
[VerifyMocks]
public class TestsThatUseTypeMock
{
}

This will clear mock expectations after each test

[TestFixture]
[ClearMocks]
public class TestsTheUseTypeMock_and_ExplicitlyVerify
{
}

Oh, and if you are asking what testing frameworks are supported, I will surprise you and tell you that all frameworks are supported!!

*Note: The Test Attributes will require a licensed edition.