A survey of software testing capabilities has revealed a dramatic increase in the number of organisations embedding testing into the software development process.
The annual report from consultancy Experimentus measured organisations testing against the independent Test Maturity Model integrated (TMMi) testing methodology.
TMMi ingrains processes into an organisation throughout software development to prevent bugs, rather than detecting them after software goes live.
The alternative to Capgemini Sogeti's Test Process Improvement (TPI) methodology has five levels (see panel below).
If carried out correctly, TMMi will find faults in software while it is being developed, rather than testing after completion. It promises to cut the costs associated with software development.
The 150 organisations in the research were measured against TMMi.
The five levels of TMMi Initial: This is basically chaos and the stage that companies are at when they have nothing in place when it comes to a testing process. These companies rely on their people, rather than processes, to find faults. Testing is done ad hoc when the software development is complete. Managed: When a company reaches this level it means testing is a core processes. This will include design, strategy, planning and setting up initial models. Testing is separated from debugging at this stage. Defined: The testing phase is no longer seen as something that happens after coding is complete. Test planning is done early on. Measured: Testing is now fully defined with measurable processes. There will also be peer reviews. Optimisation: Testing is now completely defined and improvement to processes will be made through quantitative understanding of causes of software failures.
Businesses see this type of methodology as a way to improve software and reduce faults. Software development companies can use it as a rubber stamp to show customers that they have processes in place when developing software that will reduce errors and costs as a result.
Integrated testing a growing trend
In its latest report, which questioned 150 organisations in five continents, the company found a strong increase in the number of organisations that have achieved level two and are working towards level three.
This is a sign that software testing is increasingly becoming embedded in the development lifecycle, said Geoff Thompson, chairman of the UK Testing Board and consultancy director at Experimentus.
Level two was achieved by 89% of respondents in 2011, compared with 37% in 2010 and 27.5% in 2009. This means testing is a core repeatable process and these organisations are working towards level three.
Thompson said this is very encouraging. In the past, the IT department would complete testing once the development was complete, which was costly and there would be no repeatable processes, he said, but now the CIO is putting pressure on testing terms to have measurable outcomes. Of the total sample, he said around 10% are at level three and 1-2% are at level four.
Thompson said many of the organisations putting themselves through TMMi accreditation are suppliers hoping to differentiate themselves: "Many of the companies are tier two or three suppliers that want to stand out against bigger competitors."
He said part of the reason for the sudden jump in firms that have reached level two is down to the fact that more suppliers are in the survey and more testing is being outsourced: "We have seen a trend for request for proposals (RFPs) to request the level of TMMi they want for a project." About 70% of the interviewed group were service providers.
Outsourcing software testing
More and more businesses are outsourcing software testing, and as a result, benchmarking is vital. According to analyst Nelson Hall, the global testing services market was $8.4bn in 2011, and although 2012 is expected to be flat, it predicts an average 9% growth every year over the next five years.
Thompson said end user businesses are doing it for a reference point, either to help them improve processes or use it as a way to get their development partners to meet their standards.
Working with Experimentus, UK bank HBOS carried out TMMi software across three of its four major IT departments in a programme designed to reduce the number of errors in thousands of applications it builds every year. It began using the Testing Maturity Model (TMM) in its corporate IT and insurance and investment divisions, following a successful pilot in retail banking in 2006.
Accenture is now accredited for TMMi assessment.
The Home Office is also going through the TMMi process. As a huge buyer of software development services, it can ensure the software it buys meets the right standard.
