Automated Testing in an Agile Environment

“Let’s have one final test. Throw the switches”

Recently, we needed to undertake unit testing for agile development, and continually needed to test separate modules. As we integrated the separate modules into our central repository we of course find the need to check that new additions to the repository have not altered the work already completed. Because we were verifying an embedded C system we chose to use Unity (not the game engine) for the embedded project.

More about Unity can be found at www.throwtheswitch.org and https://github.com.

To make this as efficient as possible, in the Unity test harness we created tests that consistently rerun. We do this by following design patterns, and from these produce tests scripts that are derived from an examination of the codes logic. The key techniques that we use to design test cases are:

  • Equivalence Partitioning;
  • Boundary Value Analysis (Supplementary to Equivalence Partitioning);
  • Independent Path Coverage; and
  • Decision Coverage.

The Unity test harness is a lightweight C test harness for embedded systems, and provides the tester the ability to express how the Function Under Test is required to behave during the white box testing.

Unity Test Harness

The Unity test harness provides a number of really useful features, these include:

  • The ability to express test cases and their expected results in a common language;
  • Configuration Management of the test cases and the results;
  • The flexibility to run test cases needed to provide evidence of PASS/FAIL depending on the module being considered; and
  • It provides detailed reports of the test run.

Unity provides the ability to produce assertion statements that we expect to be true about our Functions Under Test, and this supports the use of Test Driven Development.

Regression Testing and Continuous Integration

We currently conduct regression testing, this is the process of testing software to ensure that it still performs with the same expected results after further work has been carried out. With the test scripts we created it is a natural progression to use the development practice of Continuous Integration. This is an automated process which requires the recurrent integration of tested code to the stable branch of our project.

The code base contains shared resources such as mocks that provide simulate behaviour for complex system elements, definitions of global constraints, and Unity framework to support our regression testing. We found that this continuous integration approach builds confidence in the robustness of the modules under test, and where further development is essential, our team gets a really good understanding of the projects overall progression, as well as the next obstacle. It also provides confidence that the changes introduced into source control have not disturbed the function under test or the test scripts.

Lessons Learnt

This approach has the ability to reduce the cost of regression testing, and provides confidence, through continuous integration, that the final product is compliant. It goes without saying that automated testing is quicker than manual testing, allows for verification of different test permutations within a short span but requires constant tending to ensure the effectiveness of the agile verification process. This method does also require an initial investment cost, however this is soon regained.

In the context of testing using Unity, I have found that writing the tests scripts in the same language as the code under test is written in removes the need to ‘context switch’ to another tool/language and way of thinking in order to test.

I believe that this is an effective environment to continuously test any system with a clear understanding of the health of a software project in real time. We will be using this proven approach for other projects.

More From The Blog

IR35, Here it Comes Again…

IR35, Here it Comes Again…

IR35, Here it Comes Again...In 2021 the reform to IR35 Off-Payroll rules is to be rolled out to the private sector. As before the reform will only affect companies that do not meet the following attributes: an annual turnover below £10m fewer than 50 employees or a...

Solving the Resource Conundrum

Solving the Resource Conundrum

Solving the Resource ConundrumPicture this. One minute all is fine and dandy, you have access to all the resources you could possibly need, then bam an unexpected challenge arises. Suddenly you find yourself lacking the capacity to meet the new need. What are your...

Quality – An Aid to Produce Consistent Rubbish

Quality – An Aid to Produce Consistent Rubbish

Quality - An Aid to Produce Consistent RubbishAnother year has passed, and myself and a colleague have hosted a BSI auditor for our annual ISO9001/TickITplus check-up, and in fact this was more than the regular check, in that it was our 3-year re-certification audit,...

The Hazards of Legacy Systems

The Hazards of Legacy Systems

The Hazards of Legacy SystemsBeing the owner of a software system with a dedicated customer base sounds like the kind of position one would like to find themselves in. At least until it gets superseded and you have to face dealing with a legacy system. Many developers...

How to Test Without Access to The Test Environment

How to Test Without Access to The Test Environment

How to Test Without Access to The Test EnvironmentIn many of our previous articles, we have expressed the importance of achieving a high standard of testing. Potentially blocking this achievement, several factors can come together to affect the quality of your...

The Technical Workshop – How To Make Them Work For You

The Technical Workshop – How To Make Them Work For You

The Technical Workshop - How To Make Them Work For YouAnyone experienced in product design will understand just how valuable a facilitated workshop can be. Bringing together a project's key stakeholders into a single space allows for the exploration of diverse...

Developing Software for Safety Related Systems

Developing Software for Safety Related Systems

Developing Software for Safety Related SystemsSoftware systems should always be both robust and reliable, however the moment you introduce a safety element, this need for reliability increases significantly. The level of safety required is governed by the severity and...

How to Choose an Outsourcing Partner

How to Choose an Outsourcing Partner

How to Choose an Outsourcing PartnerHaving recognised a need to outsource, and worked your way through the initial preparations, you are now in a strong position to seek out a suitable partner. Choosing an outsourcing partner is no trivial affair, so taking the time...