Specification-based Testing of Reactive Software: Tools and Experiments

TitleSpecification-based Testing of Reactive Software: Tools and Experiments
Publication TypeConference Papers
Year of Publication1997
AuthorsJagadeesan L J, Porter A, Puchol C, Ramming CJ, Votta LG
Conference NameSoftware Engineering, International Conference on
Date Published1997///
PublisherIEEE Computer Society
Conference LocationLos Alamitos, CA, USA
Keywordsempirical studies, reactive systems, specification-based testing, temporal logic

Testing commercial software is expensive and time consuming. Automated testing methods promise to save a great deal of time and money throughout the software industry. One approach that is well-suited for the reactive systems found in telephone switching systems is specification-based testing.We have built a set of tools to automatically test softmare applications for violations of safety properties expressed in temporal logic. Our testing system automatically constructs finite state machine oracles corresponding to safety properties, builds test harnesses, and integrates them with the application. The test harness then generates inputs automatically to test the application.We describe a study examining the feasibility of this approach for testing industrial applications. To conduct this study we formally modeled an Automatic Protection Switching system (APS), which is an application common to many telephony systems. We then asked a number of computer science graduate students to develop several versions of the APS and use our tools to test them. We found that the tools are very effective, save significant amounts of human effort (at the expense of machine resources), and are easy to use. We also discuss improvements that are needed before we can use the tools with professional developers building commercial products.