%0 Conference Paper %B Proceedings of the 2008 international symposium on Software testing and analysis %D 2008 %T Effective and scalable software compatibility testing %A Yoon,Il-Chul %A Sussman, Alan %A Memon, Atif M. %A Porter, Adam %K component-based software system %K software compatibility testing %X Today's software systems are typically composed of multiple components, each with different versions. Software compatibility testing is a quality assurance task aimed at ensuring that multi-component based systems build and/or execute correctly across all their versions' combinations, or configurations. Because there are complex and changing interdependencies between components and their versions, and because there are such a large number of configurations, it is generally infeasible to test all potential configurations. Consequently, in practice, compatibility testing examines only a handful of default or popular configurations to detect problems; as a result costly errors can and do escape to the field. This paper presents a new approach to compatibility testing, called Rachet. We formally model the entire configuration space for software systems and use the model to generate test plans to sample a portion of the space. In this paper, we test all direct dependencies between components and execute the test plan efficiently in parallel. We present empirical results obtained by applying our approach to two large-scale scientific middleware systems. The results show that for these systems Rachet scaled well and discovered incompatibilities between components, and that testing only direct dependences did not compromise test quality. %B Proceedings of the 2008 international symposium on Software testing and analysis %S ISSTA '08 %I ACM %C New York, NY, USA %P 63 - 74 %8 2008/// %@ 978-1-60558-050-0 %G eng %U http://doi.acm.org/10.1145/1390630.1390640 %R 10.1145/1390630.1390640 %0 Conference Paper %B Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering %D 2007 %T Direct-dependency-based software compatibility testing %A Yoon,Il-Chul %A Sussman, Alan %A Memon, Atif M. %A Porter, Adam %K compatibility testing %K component-based software system %X Software compatibility testing is an important quality assurance task aimed at ensuring that component-based software systems build and/or execute properly across a broad range of user system configurations. Because each configuration can involve multiple components with different versions, and because there are complex and changing interdependencies between components and their versions, it is generally infeasible to test all potential configurations. Therefore, compatibility testing usually means examining only a handful of default or popular configurations to detect problems, and as a result costly errors can and do escape to the field This paper presents an improved approach to compatibility testing called RACHET. We formally model the configuration space for component-based systems and use the model to generate test plans covering user-specified portion of the space - the example in this paper is covering all it direct dependencies between components. The test plan is executed efficiently in parallel, by distributing work so as to best utilize test resources. We conducted experimentsand simulation studies applying our approach to a large-scale data management middleware system. The results showed that for this system RACHET discovered incompatibilities between components at a small fraction of the cost for exhaustive testing without compromising test quality %B Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering %S ASE '07 %I ACM %C New York, NY, USA %P 409 - 412 %8 2007/// %@ 978-1-59593-882-4 %G eng %U http://doi.acm.org/10.1145/1321631.1321696 %R 10.1145/1321631.1321696