Problems with NUnit 2.4 Add-In Extensions

Eli Lopian    .NET Tests    

NUnit 2.4 has a new extension feature called NUnit Add Ins. With these Add-Ins it is possible to extend NUnits capabilities without hacking the NUnit code base. This is a great step forward, although it is a feat that MBUnit has been doing for quite some time now.

The best way to understand how to write an Add-In is to download the source files and browse the examples in the samples\Extensibility\Core\ directory.
To install an Add-in copy it to the addin directory.

There is an undocumented feature that installs Add-Ins that are defined in the tested assembly, this is done by searching for an Attribute on all the Types in the assembly. The only reason that I know about this feature is that our tests failed once we upgraded to NUnit 2.4.
Our NUnit assemblies are part of the source code tree - a good practice as it allows the build to be repeatable, without requiring installations.
But NUnit kept on telling us that it couldn’t find an nunit.framework.dll (!?!). We fixed this by telling visual studio to copy the dll to the output directory (reference->Properties->Copy Local->true).
The reason that NUnit crashed, was because it was searching for the NUnit AddIn Attribute in our test assembly (and was doing this after changing the working directory)

What is completely missing from this feature is an easy way to install add-ins. Copying dll’s to addin directories simply doesn’t cut it. To Install an addin, you will have to search for all the addin directories of all nunit installations!!! 
The install.bat that is distributed with the examples should have been a big sign that this deployment strategy is flawed.
Guess what, the NUnit guys are nearly there, just extend that pesky NUnit Attribute search that you are doing for the test assembly to all referenced assembly too and viola, there is no need to install the extensions.

Perhaps because of this, Jamie’s - TestDriven.Net doesn’t support NUnit Attributes yet, rendering the extensions to more or less useless.

Best Practices, Mock Usage Pattern

Eli Lopian    .NET Tests    

There are a few ways that mocks can be used. As many of you who read my blog might guess I am not an OO Purist. I think that I am more of a Pragmatic OO. The same goes with unit tests, I don’t strongly believe that there must be one Assert per test, and I see many tests that verify the interaction and state in the same test.

But here are a few things to remember:

1. You should Verify the Mocks between tests
   This will allow the tests to fail if an expectation was not met
2. If your test fails (an exception thrown) you still need to clear your expectations
   Otherwise you will leave fake behavior for the next test

Here is a pattern that will solve these issues.

/// <summary>
/// This is the best practice tests with TypeMocks
/// </summary>
[TestFixture]
public class MyTests
{
  /// <summary>
  /// Clear mocks
  /// </summary>
  [TearDown]
  public void CleanUp()
  {
    MockManager.ClearAll();
  }

  /// <summary>
  /// We will test with mocking
  /// </summary>
  [Test]
  public void TestWithMock()
  {
     // setup mock

     // run test 

     // optional assert
     Assert.AreEqual();

     // Verify mocks
     MockManager.Verify();
  }
}

TypeMock Google Group

Eli Lopian    .NET Tests    

One of TypeMock Users has started a TypeMock Google Group.

What is interesting is the TypeMock Source Example that you can download that implements the Authentication Example described in the documentation.
Thanks, and feel free to use the group. 

TypeMock Partners and Specialist Community

Eli Lopian    .NET Tests    

I am delighted to announce that we have started a new Community for our partners and specialists.

Those in the community will have access to internal information and will have more influence on our coming features and products.
We have created a special forum for our partners, where we can meet and share information.

If you want to join this community, send me a mail (eli at typemock dot com).

TypeMock.NET Pricing

Eli Lopian    .NET Tests    

Our prices are going to rise in our next release (mid-May). All developers who are evaluating TypeMock are highly recommend to purchase before mid-May to enjoy our low prices. Existing Customers will be able to upgrade to our next release for free.

Price Changes:

• Enterprise Edition Plus 1 Year Maintenance: $375 
• Professional Edition Plus 1 Year: $275
• Annual Maintenance: $75

Mocking Safely …

Eli Lopian    .NET Tests    

There have been some concerns on the web about TypeMock being too easy, and thus dangerous.

I am really flattered with that distinction. TypeMock does make your life easy.

Here are a few tips to keep you mocking safely.

1. Test Features and not classes.
   Test features, even if they span on a few classes. Remember that you may refactor the code anyway into several classes. Once you know what features you are implementing testing, it will be easier to know what to mock.
2. Build a Mocked-Framework.
   Create a framework with your mocks, this will help when refactoring your code.
   It will also make your tests clearer

   [Test]
   public void SomeTest()
   {
      MyMockUtils.StubLogging();
      MyMockUtils.FakeCusomters();
     // continue with test
   }
   

3. Use Stubs more often
   Stubs and Conditional Expectations should be used more often
4. Refactor Your Code
   There is no excuse not to do this. Always think about how your production code is structured and refactor it.Don’t change you design just to test. There is no reason to start making private methods public and delivering code that are just for testing
5. Refactor Your Tests
   Make sure your mocks say everything once and push tests into your Mocked Framework.

A different approach to Mock Verification

Eli Lopian    .NET Tests    

Scott Bellware has posted about the awkwardness of mock testing - I find that the syntaxes for mock frameworks feel awkward to me.

It is true that the Validation of calls and arguments are hidden in the setup part of the test and the verification is run automatically at the end of the test. Here is how you can have a more explicit validation using TypeMock. (Note: I am using the Generic Syntax of our coming version)

Implicit Validation 

[Test]
public void SomeTest()
{
    // mock next instance of TestedClass this happens in SomeObject.DoYourThing();
    Mock m = Mock<TestedClass>.MockNextInstance();
    // mock foo once
    m.ExpectAndReturn("foo", "yada yada");

    // do our thing 
    SomeObject test = new SomeObject();
    test.DoYourThing();

    // verify that foo was called once
    m.Verify();
}

Explicit Validation

[Test]
public void SomeTest()
{
    // mock next instance of TestedClass this happens in SomeObject.DoYourThing();
    Mock m = Mock<TestedClass>.MockNextInstance();
    // mock foo
    m.AlwaysReturn("foo", "yada yada");

    // do our thing 
    SomeObject test = new SomeObject();
    test.DoYourThing();

    // verify that foo was called once
    Assert.AreEqual(1, m.GetCallCount("foo"));
}

There are 2 differences:

1. Using AlwaysReturn instead of ExpectAndReturn, this turns the mock into a stub.

2. Fetching and Asserting the amount of calls made, after the test for our verification.

Notice how using a remark solves Scott’s problem altogether

The downfall for using the explicit syntax are:

1. No Fast Fail – Using the explicit syntax we will not fail fast when foo was called twice (or with wrong arguments) that could lead to another failure in the test (as the code continued to run)

2. Concurrent Tests, We can’t use the concurrent features of Mocking (using VerifyWithTimeout) that enables testing multithreaded code without playing with joining threads  (just wait for a particular method in one thread to be called to signal that we can validate some data in the test thread)