For anyone who has had the pleasure of writing/maintaining tests for code that require the use of external dependencies, you will (hopefully!) have incorporated the use of a mock of a function/object. In gMock, you need to define the behaviour of a mock via the EXPECT_CALL & ON_CALL macros in each test before it is used. When multiple tests rely on the same use of a mock, the common method I see far too frequently is to either, copy-paste the mock’s behaviour across each test case that relies it, or just dump all your assertions into a single monolithic test case to avoid copy-pasting. Both are terrible habits to fall into that lead to tests that are brittle and harder to read.

In this guide, I want to demonstrate ways in which you can recognise repeated test setups, and where utilising a Test Fixture class to resolve this can simplify production & maintenance of tests.

Bad Examples of Tests

Consider the following procedural function written in C. Let’s say that we’ve been tasked to write a series of tests on some legacy code that when called retrieves some data from an external module (retrieveExternalData), before passing it on to a function that exists in another external module (sendMessage). For now, ignore the fact that this code is absolute trash and should have been refactored; it’s hard to think of a good example!

Source Code

int functionUnderTest( enum DataType data_type )
{
     int error_code;
     struct SourceDataStructure external_data;
     struct DestinationDataStructure modified_data;

    error_code = retrieveExternalData( &external_data );

     if( error_code == 0 )
    {
         // Modify data according to the data type
         switch( data_type )
        {
            case 0:
               // Modify data one way
             ...
               break;

            case 1:
               // Modify data another way
             ...
              break;

            default:
               // Return error
             error_code = -1;
              break;

     if( error_code == 0 )
      {
           error_code = sendMessage( &modified_data );
      }
    }

    return error_code;
}

In between these 2 external function calls, some manipulation of data occurs that varies depending on the value of data_type passed into this function. For now our tests will focus on testing that the data passed into ‘sendMessage’ was performed to specification according to the value of data_type.

But because of the fact that there are external dependencies that we need to mock in order to take full control over how the tests will operate, for many of the tests the mocked behaviour may be identical. As a result, different tests may exhibit the same patterns. See for example the following:

Repeating Test Cases

TEST( ModifyingData, ShallSetXWhenDataTypeIs0 )
{
      struct SourceDataStructure source_data;
      struct DestinationDataStructure output_data;
      int error_code;

      //----------------------------- ARRANGE ----------------------------
      // Configure relevant variables
    source_data.SomeProperty1 = FOO;
    source_data.SomeProperty2 = BAR;

      // Initialise output_data to invalid values
    output_data.SomeProperty1 = INVALID_VALUE;
    output_data.SomeProperty2 = INVALID_VALUE;

      // Configure the mock behaviour of retrieveExternalData
    EXPECT_CALL( *_MockObject, retrieveExternalData )
        .Times( 1 )
        .WillOnce( DoAll( SetArgPointee<0>( source_data ), // Provide the source data when it is called
                   Return( 0 ) ) ); // Return no error

      // Configure the mock behaviour of sendMessage to capture the modified data
      // and store it in output_data to allow us to assert on its properties later
    EXPECT_CALL( *_MockObject, sendMessage )
        .Times( 1 )
        .WillOnce( DoAll( SaveArg<0>( &output_data ), // Capture the modified data
                          Return( 0 ) ) ); // Return no error

      //------------------------------- ACT ------------------------------
      // Call the function under test with data type 0
    error_code = functionUnderTest( 0 );

      //------------------------------ ASSERT -----------------------------
      // Check that no error was returned
    ASSERT_THAT( error_code, 0 );

      // Check that the modified data captured is correct for this data type
    ASSERT_THAT( output_data.SomeProperty1, EXPECTED_FOR_DATATYPE_0 );
    ASSERT_THAT( output_data.SomeProperty2, EXPECTED_FOR_DATATYPE_0 );
}

TEST( ModifyingData, ShallSetXWhenDataTypeIs1 )
{
      struct SourceDataStructure source_data;
      struct DestinationDataStructure output_data;
      int error_code;

      //----------------------------- ARRANGE ----------------------------
      // Configure relevant variables
    source_data.SomeProperty1 = FOO;
    source_data.SomeProperty2 = BAR;

      // Initialise output_data to invalid values
    output_data.SomeProperty1 = INVALID_VALUE;
    output_data.SomeProperty2 = INVALID_VALUE;

      // Configure the mock behaviour of retrieveExternalData
    EXPECT_CALL( *_MockObject, retrieveExternalData )
        .Times( 1 )
        .WillOnce( DoAll( SetArgPointee<0>( source_data ), // Provide the source data when it is called
                   Return( 0 ) ) ); // Return no error

      // Configure the mock behaviour of sendMessage to capture the modified data
      // and store it in output_data to allow us to assert on its properties later
    EXPECT_CALL( *_MockObject, sendMessage )
        .Times( 1 )
        .WillOnce( DoAll( SaveArg<0>( &output_data ), // Capture the modified data
                          Return( 0 ) ) ); // Return no error

      //------------------------------- ACT ------------------------------
      // Call the function under test with data type 1
    error_code = functionUnderTest( 1 );

      //------------------------------ ASSERT -----------------------------
      // Check that no error was returned
    ASSERT_THAT( error_code, 0 );

      // Check that the modified data captured is correct for this data type
    ASSERT_THAT( output_data.SomeProperty1, EXPECTED_FOR_DATATYPE_1 );
    ASSERT_THAT( output_data.SomeProperty2, EXPECTED_FOR_DATATYPE_1 );
}

We can clearly see that these tests are almost identical in every way except for the values that we’re actually trying to test for. If the number of data types that change the behaviour of this function was to increase, this would add to the number of test cases that were needed. To make matters even worse, what happens if something changes to the external dependency that affects our code? It would probably lead to a lot of broken test cases and time would be wasted in trying to fix them all. Such pitfalls makes the process of refactoring code far more tedious than it ought to be.

Now the most natural conclusion to make when seeing code repeated would be to place that behaviour in its own function. That would be great, except for the fact that gMock will not allow the behaviour of the mocks to be defined in a function that can be called from a TEST macro. So how can this problem be solved? Thankfully, googletest provides us with…

Test Fixtures

In googletest, a Test Fixture is a class that allows us to control how a group of tests behave. We can use this to define a common behaviour for mocks and initialise data at the beginning of each test without having to continually repeat ourselves.

To begin with, the following is a bare-bones implementation of a test group derived from the TestFixture class:

TestFixture Implementation

class ModifyingData: public TestFixture
{ 
public:
      // Constructor. Allows any data that needs to be created once, but
      // used many times, to be initialised here.
    ModifyingData() : TestFixture()
    {
    }

      // Test setup. This gets called before each test is run
      void SetUp()
    {
    }

      // Test tear down. This gets called after each test has run
      void TearDown()
    {
    }
}

With this class, we can now place the default behaviours of the mocks and initialise any data within the SetUp method as follows:

TestFixture with common behaviour

class ModifyingData: public TestFixture
{ 
public:
      struct SourceDataStructure source_data;
      struct DestinationDataStructure output_data;

      // Constructor. Allows any data that needs to be created once, but
      // used many times, to be initialised here.
    ModifyingData() : TestFixture()
    {
    }

      // Test setup. This gets called before each test is run
      void SetUp()
     {
          // Configure relevant variables
        source_data.SomeProperty1 = FOO;
        source_data.SomeProperty2 = BAR;

          // Initialise output_data to invalid values
        output_data.SomeProperty1 = INVALID_VALUE;
        output_data.SomeProperty2 = INVALID_VALUE;

          // Configure the mock behaviour of retrieveExternalData
        ON_CALL( *_MockObject, retrieveExternalData )
            .WillByDefault( DoAll( SetArgPointee<0>( source_data ), // Provide the source data when it is called
                                   Return( 0 ) ) ); // Return no error

          // Configure the mock behaviour of sendMessage to capture the modified data
          // and store it in output_data to allow us to assert on its properties later
        ON_CALL( *_MockObject, sendMessage )
            .WillByDefault( DoAll( SaveArg<0>( &output_data ), // Capture the modified data
                                   Return( 0 ) ) ); // Return no error
      }

      // Test tear down. This gets called after each test has run
      void TearDown()
      {
      }
}

Now for each test run, we no longer have to define the behaviour of the mocks, nor do we have to initialise commonly used data.

You may have also noticed that the mock behaviour is using the ON_CALL method instead of the EXPECT_CALL method. The reason that this is done is that I actually do not want to actually test whether a mocked function is called by default, I just want to define how that mocked function behaves if and when it is called. If I were to use EXPECT_CALL, then I am demanding that all tests for this group must always call these functions. This may not always be the case when tests focus on control flow that may result in sendMessage never being called, causing a test to fail unnecessarily. There’s an interesting read that goes into more detail on why you would use ON_CALL over EXPECT_CALL here: Knowing When to Expect.

With this common setup out of the way, we can refactor the tests so that they are easier to read and far more maintainable. This time, all tests will use the TEST_F macro instead:

Improved Test Cases

TEST_F( ModifyingData, ShallSetXWhenDataTypeIs0 )
{
      int error_code;

      //----------------------------- ARRANGE ----------------------------
      // Nothing to do here anymore...

      //------------------------------- ACT ------------------------------
      // Call the function under test with data type 0
    error_code = functionUnderTest( 0 );

      //------------------------------ ASSERT -----------------------------
      // Check that no error was returned
    ASSERT_THAT( error_code, 0 );

      // Check that the modified data captured is correct for this data type
    ASSERT_THAT( output_data.SomeProperty1, EXPECTED_FOR_DATATYPE_0 );
    ASSERT_THAT( output_data.SomeProperty2, EXPECTED_FOR_DATATYPE_0 );
}

TEST_F( ModifyingData, ShallSetXWhenDataTypeIs1 )
{
      int error_code;

      //----------------------------- ARRANGE ----------------------------
      // Nothing to do here anymore...

      //------------------------------- ACT ------------------------------
      // Call the function under test with data type 1
    error_code = functionUnderTest( 1 );

      //------------------------------ ASSERT -----------------------------
      // Check that no error was returned
    ASSERT_THAT( error_code, 0 );

      // Check that the modified data captured is correct for this data type
    ASSERT_THAT( output_data.SomeProperty1, EXPECTED_FOR_DATATYPE_1 );
    ASSERT_THAT( output_data.SomeProperty2, EXPECTED_FOR_DATATYPE_1 );
}

Already, these tests become far easier to read, and are far more maintainable. This simplifies the process of modifying tests when changes to the source code are planned, or allows for quick updates to the behaviour of external dependencies if they are changed.

On a side note, you have probably noticed that the tests still repeat themselves. The good news is that unlike gMock’s macros, googletest’s assertion macros can be placed into a function, with parameters allowing us to define what values we want to assert against. Bad news is that a very useful feature of googletest’s Test Explorer window in Visual Studio becomes slightly less useful. For any test that fails, the Test Explorer allows you to jump to the assertion that failed. If that assertion is within a function, it jumps to the line within that function, making it more difficult to see the overall context in which it failed. However, having less code overall is more beneficial in the long term than the slight annoyance introduced by having assertions within a function. In the interest of good coding practice, let’s refactor the tests to make them even more maintainable:

Improved & Refactored Test Cases

void assertModifiedDataIsCorrect( int error_code,
                                 int expected_value_property1,
                                 int expected_value_property2 )
{
      // Check that no error was returned
    ASSERT_THAT( error_code, 0 );

      // Check that the modified data captured is correct for this data type
    ASSERT_THAT( output_data.SomeProperty1, expected_value_property1 );
    ASSERT_THAT( output_data.SomeProperty2, expected_value_property2 );
}

TEST_F( ModifyingData, ShallSetXWhenDataTypeIs0 )
{
      //----------------------------- ARRANGE ----------------------------
      // Nothing to do here anymore...

      //------------------------------- ACT ------------------------------
      //------------------------------ ASSERT -----------------------------
    assertModifiedDataIsCorrect( functionUnderTest( 0 ),
                                 EXPECTED_FOR_DATATYPE_0,
                                 EXPECTED_FOR_DATATYPE_0 ); 
}

TEST_F( ModifyingData, ShallSetXWhenDataTypeIs1 )
{
      //----------------------------- ARRANGE ----------------------------
      // Nothing to do here anymore...

      //------------------------------- ACT ------------------------------
      //------------------------------ ASSERT -----------------------------
    assertModifiedDataIsCorrect( functionUnderTest( 1 ),
                                 EXPECTED_FOR_DATATYPE_1,
                                 EXPECTED_FOR_DATATYPE_1 ); 
}

Deviating a Mock’s Default Behaviour

It should be clear by now how we can avoid repeating ourselves when defining a mock’s behaviour, but what do you do when testing for edge cases that may need a mock to do something differently? Referring back to the source code, we can observe that there is control flow within the function that will not modify any data when retrieveExternalData returns an error code other than 0:

Source Code: different control flow

error_code = retrieveExternalData( &external_data );

if( error_code == 0 )
{
      // Modify data
    ....
}

The great thing about gMock is that it allows us to create a new behaviour for a mock, and gMock will use the most recent definition that matches, allowing us to use the same Test Fixture. An example of a test case for testing this control flow could be as follows:

Overriding Default Mock Behaviour

TEST_F( ModifyingData, ShallNotOccurWhenExternalDataIsNotAvailable )
{
      int error_code;
      int expected_error_code = -1;

      //----------------------------- ARRANGE ----------------------------
      // Configure the mock behaviour of retrieveExternalData
    ON_CALL( *_MockObject, retrieveExternalData )
        .WillByDefault( Return( expected_error_code ) ) ); // Return an error

      // Ensure that data is not passed to sendMessage
    EXPECT_CALL( *_MockObject, sendMessage )
        .Times( 0 );

      //------------------------------- ACT ------------------------------
    error_code = functionUnderTest( 0 );

      //------------------------------ ASSERT -----------------------------
    ASSERT_THAT( error_code, expected_error_code ); 
}

In the above example, the mock’s behaviour in ON_CALL is defined after the one in the Test Fixture’s SetUp method, so it will take precedence over the one in SetUp.

It is also interesting that ON_CALL and EXPECT_CALL both follow this rule. This means that the most recent EXPECT_CALL can take precedence over an ON_CALL for the same mock, and vice-versa. This is useful as demonstrated in the above example, where it is important that we assert that sendMessage is never called when the function under test fails to retrieve any data from retrieveExternalData. We achieve this by specifying the cardinality of the mocked function’s call as 0 ( .Times(0) ).

Other Tests to Consider

With this knowledge, consider how the default behaviours would differ when testing the following edge cases, and whether ON_CALL or EXPECT_CALL would be appropriate:

• Passing a data_type outside of the range accounted for within this function. The function retrieveExternalData would still be called in the same way, but what happens to sendMessage?
• How could a test be written in the same Test Fixture that forces sendMessage to return an error?

When Not To Deviate Default Mock Behaviour

If you ever find yourself having to deviate a mock’s default behaviour in the same way more than once, it’s probably a good time to ask yourself whether the Test Fixture has done everything that it can reasonably do. To avoid falling into the same trap and repeating yourself, it is probably a good idea to create a new Test Fixture and defining the default behaviour for those mock(s) in its SetUp method. At the end of the day, the less code there is to maintain, the better it is for everyone. If it helps, always pretend that the person lumped with the misfortune of maintaining your code is a psychopath who knows where you live…