Monday, September 28, 2009

Advanced Software Testing using Simulation

The famous quote from Donald E. Knuth could be an interesting opening for this blog post. Unfortunately, the practice is overly cliché and would most likely dilute further the initial meaning of its author. Software testing remains a practical approach to proof softwares, whether it is manual or automated testing, whereas mathematical proofs are the theoretical approach.

Simulation can be useful to bring software testing a step further by simulating a user, or multiple users, performing a series of tasks through the software to be tested, with or without meaningful actions, but always within a certain set of parameters.

We are currently working on such simulation system. Each scenario is a compound based on a sequence of user inputs, which are collected and stored into a database. Most scenarios developed in software testing are usually based on a normal usage in order to prove the software works as intended. The existing scenarios will be subsequently mutated multiple times using probabilities, that is, the likelihood such or such user input can occur, similarly as in a Monte Carlo Simulation. Scenarios are also constructed from scratch using statistical sampling, where the value of a certain user input in a given context might or might not have a greater weight in its probability to be triggered.

The simulation system runs multiple scenarios simultaneously against multiple instances of the software during its entire development cycle on a dedicated server, on a 24/7 basis, and reports failed scenarios accordingly. The failed scenarios are then ran step-by-step by the software engineers so as to catch and correct defects.

Technologies are sometimes surprising and exciting...

1 comment:

  1. The excellent combination of low cost, supreme productivity and optimum product quality has made the west wake up to the technical strengths of the countries like India and China. These countries are making a distinct mark of their own in the field of software development by providing services like outsource software development. To know more about the services that a software company of these countries provides just check out the URL