Gaps I found while dog-fooding Typemock Isolator

I have been dog-fooding Typemock Isolator with the Metric Dashboard. There are quite a few gaps in the product that I have found while using it.

Before I go into the details of the gaps, I must point out that all these features exists in the older API’s (that are still available), but they are missing in the newer lambda-API’s (called AAA) that are much better.

Firing Events

This is not an Isolation feature per-se, but it is needed to simulate how our unit under test reacts to external events. In my case I needed to test the TestDriven.NET integration, the TestDriven add in can raise events when ever a test suite is run, and whenever a test completes (Thank Jamie).

Verifying the arguments of a constructor

One of Typemock Isolators unique features is the ability to fake objects that are created within the production code, we call these future instances. There was one case where I wanted to test that a constructor was called with a specific argument. The instance was a class that watched a specific directory for new files and load that file. I tested that class in another test, now I just wanted to test that the class was watching the correct directory. i.e. that the following was called

var watcher = new FileWatcher(testOutputDirectory);

I could of course change my code to have an empty constructor and to set the output directory after that

var watcher = new FileWatcher();
watcher.WatchFolder(testOutputDirectory);

But this felt unnatural, there is no reason that the FileWatcher be called without the directory – leading to more logic – testing that the user called WatchFolder – and complicating the application

Custom Argument Verification

Although we do have an API to verify if a call with exact argument was made, I needed to test that a specific argument was passed, but I couldn’t use the exact arguments API. The reason is that the argument was a class with 2 properties, but the Equals method was overloaded to test only one property, and I had to test both properties. I needed a custom checker. Ohad, told me that this existed but is undocumented in the NonPublic api (Api’s for private members)

Sequencing

We have put a lot of thought in the sequencing logic of the Api’s to make it easy to read and write the test on one hand, but keep the test Robust on the other (Robust as in, the test passes when the production implementation changes, but not the logic). In this test I needed to call an API to setup the data, but the date needed to be yesterday. Then with the data acting as normal, I needed to call the code and verify that a new record was added.

// pretend we started yesterday
var yesterday = DateTime.Now.AddDays(-1);
Isolate.WhenCalled(()=>DateTime.Now).WillReturn(yesterday);
var underTest = new DataModel();

// back to today
Isolate.WhenCalled(()=>DateTime.Now).WillCallOriginal();
// This should add a new record for today.
underTest.Update(); 

This didn’t work and I had to put both the Isolate lines together:

// pretend we started yesterday
var yesterday = DateTime.Now.AddDays(-1);
Isolate.WhenCalled(()=>DateTime.Now).WillReturn(yesterday);
// back to today
Isolate.WhenCalled(()=>DateTime.Now).WillCallOriginal();

var underTest = new DataModel();
// This should add a new record for today.
underTest.Update(); 

This is bad since changing the implementation to call DateTime.Now twice in the constructor will fail the test.

Assert the times a method was called

While testing the save logic, I need to test for a race condition and make sure that the save is called only once. The save logic is called from the calculation logic that is periodically called (say once a second). Once the auto-save interval is reached a save is performed in another Thread (to make sure that the application is responsive).

var saveInterval = OptionsSettings.Settings.AutoSaveEveryMinutes;
if (stopWatchForSave.Elapsed.TotalMinutes >= saveInterval)
{
  stopWatchForSave.Stop();
  ThreadPool.QueueUserWorkItem(new WaitCallback(t=>
  {
    Save();
    stopWatchForSave = Stopwatch.StartNew();
  }));
}

But if the Save takes too long and the method is called again, the save will be called again.

To test this I need to make sure that the Save method is called only once, but this API is missing.

Bonus points goes to those who know how to solve this.