Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction

TitleFault Detection Probability Analysis for Coverage-Based Test Suite Reduction
Publication TypeConference Papers
Year of Publication2007
AuthorsMcMaster S, Memon AM
Conference NameSoftware Maintenance, 2007. ICSM 2007. IEEE International Conference on
Date Published2007/10//
Keywordscoverage-based test suite reduction, fault detection probability analysis, Fault diagnosis, force coverage-based reduction, percentage fault detection reduction, percentage size reduction, program testing, software reliability, statistical analysis

Test suite reduction seeks to reduce the number of test cases in a test suite while retaining a high percentage of the original suite's fault detection effectiveness. Most approaches to this problem are based on eliminating test cases that are redundant relative to some coverage criterion. The effectiveness of applying various coverage criteria in test suite reduction is traditionally based on empirical comparison of two metrics derived from the full and reduced test suites and information about a set of known faults: (1) percentage size reduction and (2) percentage fault detection reduction, neither of which quantitatively takes test coverage data into account. Consequently, no existing measure expresses the likelihood of various coverage criteria to force coverage-based reduction to retain test cases that expose specific faults. In this paper, we develop and empirically evaluate, using a number of different coverage criteria, a new metric based on the "average expected probability of finding a fault" in a reduced test suite. Our results indicate that the average probability of detecting each fault shows promise for identifying coverage criteria that work well for test suite reduction.