An experiment to assess the cost-benefits of code inspections in large scale software development

TitleAn experiment to assess the cost-benefits of code inspections in large scale software development
Publication TypeJournal Articles
Year of Publication1997
AuthorsPorter A, Siy HP, Toman CA, Votta LG
JournalIEEE Transactions on Software Engineering
Pagination329 - 346
Date Published1997/06//
ISBN Number0098-5589
KeywordsAnalysis of variance, code inspection cost benefits, code unit, commercial software product, Computer Society, cost-benefit analysis, Costs, defect detection effectiveness, defect detection rate, Design optimization, experienced software developers, experiment design, independent variables, Inspection, inspection interval, inspection team, Laboratories, large scale software development, Large-scale systems, live development effort, long term experiment, professional aspects, professional developers, Programming, reviewers, simulation techniques, software cost estimation, software inspection methods, software practitioners, Software quality, Switches

We conducted a long term experiment to compare the costs and benefits of several different software inspection methods. These methods were applied by professional developers to a commercial software product they were creating. Because the laboratory for this experiment was a live development effort, we took special care to minimize cost and risk to the project, while maximizing our ability to gather useful data. The article has several goals: (1) to describe the experiment's design and show how we used simulation techniques to optimize it; (2) to present our results and discuss their implications for both software practitioners and researchers; and (3) to discuss several new questions raised by our findings. For each inspection, we randomly assigned three independent variables: (1) the number of reviewers on each inspection team (1, 2, or 4); (2) the number of teams inspecting the code unit (1 or 2); and (3) the requirement that defects be repaired between the first and second team's inspections. The reviewers for each inspection were randomly selected without replacement from a pool of 11 experienced software developers. The dependent variables for each inspection included inspection interval (elapsed time), total effort, and the defect detection rate. Our results showed that these treatments did not significantly influence the defect detection effectiveness, but that certain combinations of changes dramatically increased the inspection interval