Speaker
Mr
Wijnands Quirien
(ESA/Software Systems Division)
Description
Activity: Laboratory (R&D)
TO: Quirien Wijnands
Software Systems’ Testing and Verification is one of the major phases in a software development life-cycle. The Testing and Verification is a process that must be closely aligned with the Design and Development of that same software system and should start right from the very beginning of the project. In doing so, the definition of testcases, their execution and the analysis of test results is a time-consuming and costly part of the software development process and in almost all cases is underestimated at the start of the project. In cases where the design, development, tests definitions and tests execution are distributed over different persons the process becomes even harder since knowledge about the system needs to be transferred (e.g. what are the most critical or complex elements of the software).
Ideally, after making a modification/updates to a software module, all test cases could be rerun. However in most cases, due to limited resources (e.g. hardware availability) or limited time, this is not possible and only a subset of tests can be executed. Prioritisation of the test cases to be executed is a complex task (e.g. based on complexity and number of software modules, number of changes to the software etc.). As a consequence important testcases could be overlooked and “useless” ones are executed. In many cases, in reducing the cost, test campaigns just follow a predefined list of procedures without taking into account the knowledge and responses of the system under test at all.
Testing and Verification requires expertise based on the knowledge of the system under test, but at the same time can be a very repetitive task. Agent-based expert systems can emulate this human expertise, though in well-defined problem domains. Using a knowledge-base and rule-based behaviours, a set of agents can not only automate the testing process but also bring “intelligence” into the Testing and Verification process. For example, based on test results the order of the test scenario could be modified. In this way the Testing and Verification process can be enhanced, the possibility of human errors can be reduced and a cost reduction in the long run can be foreseen.
Using the concept of Agents as defined in the DAFA studies, in the MASTV activity an analysis was done on the usefulness of agents during the testing phase of a Software System.
In doing so the concept of "Software Quality" was used, how to characterize this and how to measure it. Based on this analysis and an identified Test Process a first design of the Multi Agent System was created.
Out of this some components (agents) were modelled and implemented as part of a software prototype.
