Can We Trust Humans to Test Software?

Many of you will agree that testing software is not the most exciting or interesting of tasks. There are some amongst us that do enjoy the challenge of identifying test cases that will completely exercise the module, task, sub-system or system and good for them. But mostly companies cannot afford the luxury of having a dedicated test team and the burden of testing falls to members of the development team. Those that draw the short straw!

A developer that has been working in the heady realms of web-base 3D graphics is not going to have ‘his boat floated’ by testing serial comms on an embedded processor. That may be a bit of an extreme example, but you get the point.

So what is probably the most important task before system delivery is entrusted to a group of de-motivated engineers that just want to finish as quickly as possible? Testing is repetitive, tedious and difficult. Add to that the pressures of looming delivery dates and soaring costs, and it’s a recipe for warrant work; or reduced profit.

This is true of all stages of testing but experience shows that regression testing falls into this trap the most. So many companies have ‘products’ that are really a core of standard software that then has to be modified to meet the requirements of a current application. The new software is developed and tested, but then the core software needs to be checked to ensure that it still does what it’s supposed to. This is a real bore, it’s all been done before but it’s vital in order to maintain profitability and reputation.

System testing where links to other systems need to be simulated to produce valid and invalid inputs, is another area where it can be tedious to generate and apply test cases. Again, this area has a big impact on profit and reputation.

Automation is the key. It obviously involves an initial investment of time and money, but the ROI can be enormous. Building a repeatable set of tests to exercise all of the functionality and the error checking is not a simple task BUT it must be done if you are to be confident that the system will not fail following installation. Once the tests have been generated they can be saved along with the software and can be reused to investigate problems or as a regression test for the next iteration of the software.

This is a problem that we have come across many times and are developing a tool to overcome. Look for ARTIST on our website soon.