Saturday, 29 January 2011

Are you a Mockist or a Classicist?

I recently re-read Martin Fowler’s essay ‘Mocks aren’t Stubs’ comparing the benefits and pitfalls of writing unit tests using either state verification or behavioural verification. He describes state verification as classical or regular unit testing, in that you run a test and check the state of your test object (as defined by the data a particular method call returns) using some kind of assert(…) method.

A Classic JUnit Test

public void testFlowStats1() {
    final int loopVal = 51;
    // Set the object up
    for (int i = 0; i < loopVal; i++) {
    // Call the method under test
    TimerStatistics.TimerResults result = instance.calculateStats();
    // Verify the results 
    assertEquals(result.getAvgRecentIntervalTime(), result.getAvgTotalInterval(), 0);
    assertEquals(loopVal - 1, result.getDataPoints());
    assertEquals(loopVal - 1, result.getRecentCountSize());

Mockists, on the other hand, write unit tests that test the behaviour of code by describing a path through the code, calling the method under test and verifying that the code actually took the described path.

A Mockist JUnit Test

 * This test uses Easymock and Unitils
public void testManagerStartSequenceTaskCompleted() throws ExecutionException {

    // setup methods called by run()

    // Setup internal state
    CallReturnBean retBean = new CallReturnBean();


    // call the instance and run the test;

Reading the essay, I get the feeling that you’re supposed to choose: Mockist or Classicist, it's a bit like being either Roundhead or Cavilier, or choosing between Laural or Hardy; Morecambe or Wise; Lennon or McCartney.

Personally, I believe that these two approaches compliment each other; hence, I tend to be neither a Mockist nor Classicist, but more of a pragmatist. I like the testing isolation provided by mock objects, but also like to ensure that my classes work with their collaborators, which means using some classical state validation tests.

So, my current unit test strategy is to write a large number of mock based tests that test every route through the code, to ensure all scenarios work. These are highly isolated, fine grained tests that run very quickly and should be run most often.

When I’m confident that my class works, I also like to add a smaller number of classical or state verification tests that ensure my class works with its collaborators. This is doubly important if you’re using Spring to create a dependency injection network as it’s much quicker to load a Spring configuration on your local machine rather than to test it by deploying to your JEE server. Some would classify these coarser grained tests as mini-integration tests.

There are times, however, when mocks don’t provide the best solution and when classical tests come to in to their own. For example, in an enterprise application, at some point you’ll need a few tests that ensure your application can talk to the database and that it can read and write data appropriately.

Another case where mocks seem inappropriate is when your instance under test performs some kind of calculation, for example it may calculate some statistics. In this case you’ll need to check the result returned by your method, rather than the route taken through the code.

So, as I said, programming tests should come down to pragmatism, doing what’s simplest and easiest in order to get the code written and delivered.


Anonymous said...

Could you please explain how not to be a mockist where the test subject depends on any other object?

How can I test an object without mocking its dependencies: e.g. if there is a service you are testing, you either have to insert a working DAO ot a mock DAO but if you are a classiscist, and only "check the state of your test object", you will get NPEs because the calls the service makes do not bear any results.

RogHughe said...

There is a difference between Mock objects and Stub objects, which personally comes down to implementation semantics.

A Mockist would inject a Mock object and a Classicist would use a stub.

A mock object is an object in which you can define a path through your code and verify that the path has been followed, i.e. you set expectations and verify that they have occurred.

A stub, on the other hand, is a dummy object, which when accessed will return dummy values to your code. You cannot verify that a stub object has been called, just that your code under test return the correct result.

So, you have the choice of which type of fake implementation you inject.

logistiker said...

Why the mini- prefix before integration tests? Just admit that you writing and performing a full fledged integration test and not a unit test. You cannot call it a unit test if it does not remain in isolation and calling a unit test a mini-integration test is like calling an apple an orange. The defition of both levels of testing says it all:

Roger Hughes said...

Logisiker, thank you for your comment. I agree that the term 'mini-integration test' isn't a particularly good one. I used it to try to differentiate between integration testing small sets of classes, which would, for example be part of the bottom up approach described by Wikipedia, and end to end integration tests. The term 'integration test' so vague that it has been applied almost any test where more than one class is involved. I often think that we should come up with a set of names that accurately describe the differences between the different types of integration tests, at least that way I wouldn't feel then need to make one up.