%0 Conference Paper %B Proceedings of the 24th annual ACM symposium on User interface software and technology %D 2011 %T Associating the visual representation of user interfaces with their internal structures and metadata %A Chang,Tsung-Hsiang %A Tom Yeh %A Miller,Rob %K accessibility api %K Graphical user interfaces %K Pixel %K text detection %K text segmentation %X Pixel-based methods are emerging as a new and promising way to develop new interaction techniques on top of existing user interfaces. However, in order to maintain platform independence, other available low-level information about GUI widgets, such as accessibility metadata, was neglected intentionally. In this paper, we present a hybrid framework, PAX, which associates the visual representation of user interfaces (i.e. the pixels) and their internal hierarchical metadata (i.e. the content, role, and value). We identify challenges to building such a framework. We also develop and evaluate two new algorithms for detecting text at arbitrary places on the screen, and for segmenting a text image into individual word blobs. Finally, we validate our framework in implementations of three applications. We enhance an existing pixel-based system, Sikuli Script, and preserve the readability of its script code at the same time. Further, we create two novel applications, Screen Search and Screen Copy, to demonstrate how PAX can be applied to development of desktop-level interactive systems. %B Proceedings of the 24th annual ACM symposium on User interface software and technology %S UIST '11 %I ACM %C New York, NY, USA %P 245 - 256 %8 2011/// %@ 978-1-4503-0716-1 %G eng %U http://doi.acm.org/10.1145/2047196.2047228 %R 10.1145/2047196.2047228 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2011 %T Developing a Single Model and Test Prioritization Strategies for Event-Driven Software %A Bryce,R.C. %A Sampath,S. %A Memon, Atif M. %K EDS %K event-driven software %K graphical user interface %K Graphical user interfaces %K GUI testing %K Internet %K program testing %K service-oriented architecture %K test prioritization strategy %K Web application testing %X Event-Driven Software (EDS) can change state based on incoming events; common examples are GUI and Web applications. These EDSs pose a challenge to testing because there are a large number of possible event sequences that users can invoke through a user interface. While valuable contributions have been made for testing these two subclasses of EDS, such efforts have been disjoint. This work provides the first single model that is generic enough to study GUI and Web applications together. In this paper, we use the model to define generic prioritization criteria that are applicable to both GUI and Web applications. Our ultimate goal is to evolve the model and use it to develop a unified theory of how all EDS should be tested. An empirical study reveals that the GUI and Web-based applications, when recast using the new model, show similar behavior. For example, a criterion that gives priority to all pairs of event interactions did well for GUI and Web applications; another criterion that gives priority to the smallest number of parameter value settings did poorly for both. These results reinforce our belief that these two subclasses of applications should be modeled and studied together. %B Software Engineering, IEEE Transactions on %V 37 %P 48 - 64 %8 2011/// %@ 0098-5589 %G eng %N 1 %R 10.1109/TSE.2010.12 %0 Conference Paper %B 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) %D 2011 %T Evaluating visual and statistical exploration of scientific literature networks %A Gove,R. %A Dunne,C. %A Shneiderman, Ben %A Klavans,J. %A Dorr, Bonnie J %K abstracting %K academic literature %K action science explorer %K automatic clustering %K citation analysis %K citation network visualization %K Communities %K Context %K custom exploration goal %K Data visualization %K Databases %K Document filtering %K document handling %K document ranking %K easy-to-understand metrics %K empirical evaluation %K Google %K Graphical user interfaces %K Information filtering %K Information Visualization %K Libraries %K literature exploration %K network statistics %K paper filtering %K paper ranking %K scientific literature network %K statistical exploration %K summarization technique %K user-defined tasks %K visual exploration %K Visualization %X Action Science Explorer (ASE) is a tool designed to support users in rapidly generating readily consumable summaries of academic literature. It uses citation network visualization, ranking and filtering papers by network statistics, and automatic clustering and summarization techniques. We describe how early formative evaluations of ASE led to a mature system evaluation, consisting of an in-depth empirical evaluation with four domain experts. The evaluation tasks were of two types: predefined tasks to test system performance in common scenarios, and user-defined tasks to test the system's usefulness for custom exploration goals. The primary contribution of this paper is a validation of the ASE design and recommendations to provide: easy-to-understand metrics for ranking and filtering documents, user control over which document sets to explore, and overviews of the document set in coordinated views along with details-on-demand of specific papers. We contribute a taxonomy of features for literature search and exploration tools and describe exploration goals identified by our participants. %B 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) %I IEEE %P 217 - 224 %8 2011/09/18/22 %@ 978-1-4577-1246-3 %G eng %R 10.1109/VLHCC.2011.6070403 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2011 %T GUI Interaction Testing: Incorporating Event Context %A Xun Yuan %A Cohen,M. B %A Memon, Atif M. %K automatic test case generation %K automatic test pattern generation %K combinatorial interaction testing %K event driven nature %K graphical user interface %K Graphical user interfaces %K GUI interaction testing %K program testing %X Graphical user interfaces (GUIs), due to their event-driven nature, present an enormous and potentially unbounded way for users to interact with software. During testing, it is important to #x201C;adequately cover #x201D; this interaction space. In this paper, we develop a new family of coverage criteria for GUI testing grounded in combinatorial interaction testing. The key motivation of using combinatorial techniques is that they enable us to incorporate #x201C;context #x201D; into the criteria in terms of event combinations, sequence length, and by including all possible positions for each event. Our new criteria range in both efficiency (measured by the size of the test suite) and effectiveness (the ability of the test suites to detect faults). In a case study on eight applications, we automatically generate test cases and systematically explore the impact of context, as captured by our new criteria. Our study shows that by increasing the event combinations tested and by controlling the relative positions of events defined by the new criteria, we can detect a large number of faults that were undetectable by earlier techniques. %B Software Engineering, IEEE Transactions on %V 37 %P 559 - 574 %8 2011/08//july %@ 0098-5589 %G eng %N 4 %R 10.1109/TSE.2010.50 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2010 %T Generating Event Sequence-Based Test Cases Using GUI Runtime State Feedback %A Xun Yuan %A Memon, Atif M. %K automatic model driven technique %K event interaction coverage equivalent counterparts %K event semantic interaction relationships %K event sequence based test cases %K Graphical user interfaces %K GUI runtime state feedback %K program testing %K Software quality %X This paper presents a fully automatic model-driven technique to generate test cases for graphical user interfaces (GUIs)-based applications. The technique uses feedback from the execution of a ??seed test suite,?? which is generated automatically using an existing structural event interaction graph model of the GUI. During its execution, the runtime effect of each GUI event on all other events pinpoints event semantic interaction (ESI) relationships, which are used to automatically generate new test cases. Two studies on eight applications demonstrate that the feedback-based technique 1) is able to significantly improve existing techniques and helps identify serious problems in the software and 2) the ESI relationships captured via GUI state yield test suites that most often detect more faults than their code, event, and event-interaction-coverage equivalent counterparts. %B Software Engineering, IEEE Transactions on %V 36 %P 81 - 95 %8 2010/02//jan %@ 0098-5589 %G eng %N 1 %R 10.1109/TSE.2009.68 %0 Conference Paper %B Software Testing, Verification and Validation (ICST), 2010 Third International Conference on %D 2010 %T Repairing GUI Test Suites Using a Genetic Algorithm %A Huang,Si %A Cohen,M. B %A Memon, Atif M. %K automated functional testing %K genetic algorithm %K Genetic algorithms %K graph model %K graphical user interface %K Graphical user interfaces %K GUI test suite %K program testing %K random algorithm %K synthetic program %K test case %X Recent advances in automated functional testing of Graphical User Interfaces (GUIs) rely on deriving graph models that approximate all possible sequences of events that may be executed on the GUI, and then use the graphs to generate test cases (event sequences) that achieve a specified coverage goal. However, because these models are only approximations of the actual event flows, the generated test cases may suffer from problems of infeasibility, i.e., some events may not be available for execution causing the test case to terminate prematurely. In this paper we develop a method to automatically repair GUI test suites, generating new test cases that are feasible. We use a genetic algorithm to evolve new test cases that increase our test suite's coverage while avoiding infeasible sequences. We experiment with this algorithm on a set of synthetic programs containing different types of constraints and for test sequences of varying lengths. Our results suggest that we can generate new test cases to cover most of the feasible coverage and that the genetic algorithm outperforms a random algorithm trying to achieve the same goal in almost all cases. %B Software Testing, Verification and Validation (ICST), 2010 Third International Conference on %P 245 - 254 %8 2010/04// %G eng %R 10.1109/ICST.2010.39 %0 Conference Paper %B 2010 IEEE Workshop on Signal Processing Systems (SIPS) %D 2010 %T Scalable representation of dataflow graph structures using topological patterns %A Sane, N. %A Kee, Hojin %A Seetharaman, G. %A Bhattacharyya, Shuvra S. %K arrays %K data flow graphs %K Dataflow graphs %K DIF language %K Digital signal processing %K directed graphs %K DSP specification languages %K embedded signal processing design flows %K embedded systems %K Field programmable gate arrays %K graphical user interface %K Graphical user interfaces %K High-level languages %K modelbased design %K optimisation %K optimizations %K scalable dataflow graph structures representation %K semantics %K Signal processing %K Signal processing systems %K Specification languages %K text based languages %K Topological patterns %K Topology %K Transform coding %X Tools for designing signal processing systems with their semantic foundation in dataflow modeling often use high-level graphical user interface (GUI) or text based languages that allow specifying applications as directed graphs. Such graphical representations serve as an initial reference point for further analysis and optimizations that lead to platform-specific implementations. For large-scale applications, the underlying graphs often consist of smaller substructures that repeat multiple times. To enable more concise representation and direct analysis of such substructures in the context of high level DSP specification languages and design tools, we develop the modeling concept of topological patterns, and propose ways for supporting this concept in a high-level language. We augment the DIF language - a language for specifying DSP-oriented dataflow graphs - with constructs for supporting topological patterns, and we show how topological patterns can be effective in various aspects of embedded signal processing design flows using specific application examples. %B 2010 IEEE Workshop on Signal Processing Systems (SIPS) %P 13 - 18 %8 2010 %G eng %0 Conference Paper %B Software Testing, Verification, and Validation Workshops (ICSTW), 2010 Third International Conference on %D 2010 %T Using methods & measures from network analysis for gui testing %A Elsaka,E. %A Moustafa,W. E %A Nguyen,Bao %A Memon, Atif M. %K betweenness clustering method %K event sequences %K event-flow graph model %K Graphical user interfaces %K GUI quality assurance %K GUI testing %K network analysis %K network centrality measures %K program testing %K Software quality %X Graphical user interfaces (GUIs) for today's applications are extremely large. Moreover, they provide many degrees of freedom to the end-user, thus allowing the user to perform a very large number of event sequences on the GUI. The large sizes and degrees of freedom create severe problems for GUI quality assurance, including GUI testing. In this paper, we leverage methods and measures from network analysis to analyze and study GUIs, with the goal of aiding GUI testing activities. We apply these methods and measures on the event-flow graph model of GUIs. Results of a case study show that "network centrality measures" are able to identify the most important events in the GUI as well as the most important sequences of events. These events and sequences are good candidates for test prioritization. In addition, the "betweenness clustering" method is able to partition the GUI into regions that can be tested separately. %B Software Testing, Verification, and Validation Workshops (ICSTW), 2010 Third International Conference on %P 240 - 246 %8 2010/04// %G eng %R 10.1109/ICSTW.2010.61 %0 Conference Paper %B Software Testing, Verification and Validation Workshops, 2009. ICSTW '09. International Conference on %D 2009 %T An Extensible Heuristic-Based Framework for GUI Test Case Maintenance %A McMaster,S. %A Memon, Atif M. %K extensible heuristic-based framework %K graphical user interface %K Graphical user interfaces %K GUI code %K GUI test case maintenance %K program testing %K Software maintenance %X Graphical user interfaces (GUIs) make up a large portion of the code comprising many modern software applications. However, GUI testing differs significantly from testing of traditional software. One respect in which this is true is test case maintenance. Due to the way that GUI test cases are often implemented, relatively minor changes to the construction of the GUI can cause a large number of test case executions to malfunction, often because GUI elements referred to by the test cases have been renamed, moved, or otherwise altered. We posit that a general solution to the problem of GUI test case maintenance must be based on heuristics that attempt to match an applicationpsilas GUI elements across versions. We demonstrate the use of some heuristics with framework support. Our tool support is general in that it may be used with other heuristics if needed in the future. %B Software Testing, Verification and Validation Workshops, 2009. ICSTW '09. International Conference on %P 251 - 254 %8 2009/04// %G eng %R 10.1109/ICSTW.2009.11 %0 Conference Paper %B Software Testing Verification and Validation, 2009. ICST '09. International Conference on %D 2009 %T An Initial Characterization of Industrial Graphical User Interface Systems %A Brooks,P.A. %A Robinson,B.P. %A Memon, Atif M. %K Graphical user interfaces %K GUI-based software systems %K industrial graphical user interface systems %K model-based GUI testing techniques %K program testing %K software metrics %K source code change metrics %X To date we have developed and applied numerous model-based GUI testing techniques; however, we are unable to provide definitive improvement schemes to real-world GUI test planners, as our data was derived from open source applications, small compared to industrial systems. This paper presents a study of three industrial GUI-based software systems developed at ABB, including data on classified defects detected during late-phase testing and customer usage, test suites, and source code change metrics. The results show that (1) 50% of the defects found through the GUI are categorized as data access and handling, control flow and sequencing, correctness, and processing defects, (2) system crashes exposed defects 12-19% of the time, and (3) GUI and non-GUI components are constructed differently, in terms of source code metrics. %B Software Testing Verification and Validation, 2009. ICST '09. International Conference on %P 11 - 20 %8 2009/04// %G eng %R 10.1109/ICST.2009.11 %0 Conference Paper %B Software Testing, Verification and Validation Workshops, 2009. ICSTW '09. International Conference on %D 2009 %T Towards Dynamic Adaptive Automated Test Generation for Graphical User Interfaces %A Xun Yuan %A Cohen,M. B %A Memon, Atif M. %K adaptive automated test generation %K computational complexity %K event sequence length %K evolutionary algorithm %K evolutionary computation %K graphical user interface %K Graphical user interfaces %K GUI test case %K program testing %X Graphical user interfaces (GUIs) present an enormous number of potential event sequences to users. During testing it is necessary to cover this space, however the complexity of modern GUIs has made this an increasingly difficult task. Our past work has demonstrated that it is important to incorporate "context” into GUI test cases, in terms of event combinations, event sequence length, and by considering all possible starting and ending positions for each event. Despite the use of our most refined modeling techniques, many of the generated test cases remain unexecutable. In this paper, we posit that due to the dynamic state-based nature of GUIs, it is important to incorporate feedback from the execution of tests into test case generation algorithms. We propose the use of an evolutionary algorithm to generate test suites with fewer unexecutable test cases and higher event interaction coverage. %B Software Testing, Verification and Validation Workshops, 2009. ICSTW '09. International Conference on %P 263 - 266 %8 2009/04// %G eng %R 10.1109/ICSTW.2009.26 %0 Conference Paper %B Practice and Research Techniques, 2008. TAIC PART '08. Testing: Academic Industrial Conference %D 2008 %T Alternating GUI Test Generation and Execution %A Xun Yuan %A Memon, Atif M. %K ALT %K complex failure-causing interactions %K event sequences %K graphical user interface %K Graphical user interfaces %K GUI test generation %K Testing %X Users of today's software perform tasks by interacting with a graphical user interface (GUI) front-end via sequences of input events. Due to the flexibility offered by most GUIs, the number of event sequences grows exponentially with length. One ubiquitous challenge of GUI testing is to selectively generate those sequences that lead to potentially problematic states. This paper presents ALT, a new technique that generates GUI test cases in batches, by leveraging GUI run-time information from a previously run batch to obtain the next batch. Each successive batch consists of "longer" test cases that expand the state space to be explored, yet prune the "unimportant" states. The "alternating" nature of ALT allows it to enhance the next batch by leveraging certain relationships between GUI events (e.g., one enables the other, one alters the other's execution) that are revealed only at run-time and non-trivial to infer statically. An empirical study on four fielded GUI-based applications demonstrates that ALT is successful at identifying complex failure-causing interactions between GUI events. %B Practice and Research Techniques, 2008. TAIC PART '08. Testing: Academic Industrial Conference %P 23 - 32 %8 2008/// %G eng %R 10.1109/TAIC-PART.2008.10 %0 Journal Article %J ACM Transactions on Software Engineering and Methodology (TOSEM) %D 2008 %T Automatically repairing event sequence-based GUI test suites for regression testing %A Memon, Atif M. %K Graphical user interfaces %K regression testing %K repairing test cases %K test case management %K test maintenance %X Although graphical user interfaces (GUIs) constitute a large part of the software being developed today and are typically created using rapid prototyping, there are no effective regression testing techniques for GUIs. The needs of GUI regression testing differ from those of traditional software. When the structure of a GUI is modified, test cases from the original GUI's suite are either reusable or unusable on the modified GUI. Because GUI test case generation is expensive, our goal is to make the unusable test cases usable, thereby helping to retain the suite's event coverage. The idea of reusing these unusable (obsolete) test cases has not been explored before. This article shows that a large number of test cases become unusable for GUIs. It presents a new GUI regression testing technique that first automatically determines the usable and unusable test cases from a test suite after a GUI modification, then determines the unusable test cases that can be repaired so that they can execute on the modified GUI, and finally uses repairing transformations to repair the test cases. This regression testing technique along with four repairing transformations has been implemented. An empirical study for four open-source applications demonstrates that (1) this approach is effective in that many of the test cases can be repaired, and is practical in terms of its time performance, (2) certain types of test cases are more prone to becoming unusable, and (3) certain types of “dominator” events, when modified, make a large number of test cases unusable. %B ACM Transactions on Software Engineering and Methodology (TOSEM) %V 18 %P 4:1–4:36 - 4:1–4:36 %8 2008/11// %@ 1049-331X %G eng %U http://doi.acm.org/10.1145/1416563.1416564 %N 2 %R 10.1145/1416563.1416564 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2008 %T Call-Stack Coverage for GUI Test Suite Reduction %A McMaster,S. %A Memon, Atif M. %K call-stack coverage criterion %K computer testing %K Graphical user interfaces %K test suite reduction %X Graphical user interfaces (GUIs) are used as front ends to most of today's software applications. The event-driven nature of GUIs presents new challenges for testing. One important challenge is test suite reduction. Conventional reduction techniques/tools based on static analysis are not easily applicable due to the increased use of multilanguage GUI implementations, callbacks for event handlers, virtual function calls, reflection, and multithreading. Moreover, many existing techniques ignore code in libraries and fail to consider the context in which event handlers execute. Consequently, they yield GUI test suites with seriously impaired fault-detection abilities. This paper presents a reduction technique based on the call-stack coverage criterion. Call stacks may be collected for any executing program with very little overhead. Empirical studies in this paper compare reduction based on call-stack coverage to reduction based on line, method, and event coverage, including variations that control for the size and optional consideration of library methods. These studies show that call-stack-based reduction provides unique trade-offs between the reduction in test suite size and the loss of fault detection effectiveness, which may be valuable in practice. Additionally, an analysis of the relationship between coverage requirements and fault-revealing test cases is presented. %B Software Engineering, IEEE Transactions on %V 34 %P 99 - 115 %8 2008/02//jan %@ 0098-5589 %G eng %N 1 %R 10.1109/TSE.2007.70756 %0 Journal Article %J IEEE Transactions on Visualization and Computer Graphics %D 2008 %T Interactive Entity Resolution in Relational Data: A Visual Analytic Tool and Its Evaluation %A Kang,Hyunmo %A Getoor, Lise %A Shneiderman, Ben %A Bilgic,M. %A Licamele,L. %K algorithms %K Computer Graphics %K D-Dupe %K data visualisation %K database management systems %K Databases, Factual %K graphical user interface %K Graphical user interfaces %K human-centered computing %K Image Interpretation, Computer-Assisted %K Information Storage and Retrieval %K Information Visualization %K interactive entity resolution %K relational context visualization %K Relational databases %K relational entity resolution algorithm %K User interfaces %K user-centered design %K User-Computer Interface %K visual analytic tool %X Databases often contain uncertain and imprecise references to real-world entities. Entity resolution, the process of reconciling multiple references to underlying real-world entities, is an important data cleaning process required before accurate visualization or analysis of the data is possible. In many cases, in addition to noisy data describing entities, there is data describing the relationships among the entities. This relational data is important during the entity resolution process; it is useful both for the algorithms which determine likely database references to be resolved and for visual analytic tools which support the entity resolution process. In this paper, we introduce a novel user interface, D-Dupe, for interactive entity resolution in relational data. D-Dupe effectively combines relational entity resolution algorithms with a novel network visualization that enables users to make use of an entity's relational context for making resolution decisions. Since resolution decisions often are interdependent, D-Dupe facilitates understanding this complex process through animations which highlight combined inferences and a history mechanism which allows users to inspect chains of resolution decisions. An empirical study with 12 users confirmed the benefits of the relational context visualization on the performance of entity resolution tasks in relational data in terms of time as well as users' confidence and satisfaction. %B IEEE Transactions on Visualization and Computer Graphics %V 14 %P 999 - 1014 %8 2008/10//Sept %@ 1077-2626 %G eng %N 5 %R 10.1109/TVCG.2008.55 %0 Conference Paper %B Software Testing, Verification, and Validation, 2008 1st International Conference on %D 2008 %T Relationships between Test Suites, Faults, and Fault Detection in GUI Testing %A Strecker,J. %A Memon, Atif M. %K Fault detection %K fault-related factors %K Graphical user interfaces %K GUI testing %K program testing %K software-testing %K test suites %K test-suite-related factors %X Software-testing researchers have long sought recipes for test suites that detect faults well. In the literature, empirical studies of testing techniques abound, yet the ideal technique for detecting the desired kinds of faults in a given situation often remains unclear. This work shows how understanding the context in which testing occurs, in terms of factors likely to influence fault detection, can make evaluations of testing techniques more readily applicable to new situations. We present a methodology for discovering which factors do statistically affect fault detection, and we perform an experiment with a set of test-suite- and fault-related factors in the GUI testing of two fielded, open-source applications. Statement coverage and GUI-event coverage are found to be statistically related to the likelihood of detecting certain kinds of faults. %B Software Testing, Verification, and Validation, 2008 1st International Conference on %P 12 - 21 %8 2008/04// %G eng %R 10.1109/ICST.2008.26 %0 Conference Paper %B Companion of the 30th international conference on Software engineering %D 2008 %T Test case generator for GUITAR %A Hackner,Daniel R. %A Memon, Atif M. %K code coverage %K event-flow %K Graphical user interfaces %K gui ripping %K integration tree %K jfcunit %K junit %K Test Case Generation %X As GUIs become more popular, the need for GUI testing tools becomes greater. Many current GUI test generation techniques require proprietary tools and can be hard to use to their fullest potential. This paper outlines a new test case generation strategy, which enables testers to automatically produce cases in a widely used format. We hope that this strategy will encourage more complete GUI testing. %B Companion of the 30th international conference on Software engineering %S ICSE Companion '08 %I ACM %C New York, NY, USA %P 959 - 960 %8 2008/// %@ 978-1-60558-079-1 %G eng %U http://doi.acm.org/10.1145/1370175.1370207 %R 10.1145/1370175.1370207 %0 Journal Article %J ACM Transactions on Software Engineering and Methodology (TOSEM) %D 2008 %T Using a pilot study to derive a GUI model for automated testing %A Xie,Qing %A Memon, Atif M. %K Graphical user interfaces %K model-based testing %K test minimization %K Test suite management %X Graphical user interfaces (GUIs) are one of the most commonly used parts of today's software. Despite their ubiquity, testing GUIs for functional correctness remains an understudied area. A typical GUI gives many degrees of freedom to an end-user, leading to an enormous input event interaction space that needs to be tested. GUI test designers generate and execute test cases (modeled as sequences of user events) to traverse its parts; targeting a subspace in order to maximize fault detection is a nontrivial task. In this vein, in previous work, we used informal GUI code examination and personal intuition to develop an event-interaction graph (EIG). In this article we empirically derive the EIG model via a pilot study, and the resulting EIG validates our intuition used in previous work; the empirical derivation process also allows for model evolution as our understanding of GUI faults improves. Results of the pilot study show that events interact in complex ways; a GUI's response to an event may vary depending on the context established by preceding events and their execution order. The EIG model helps testers to understand the nature of interactions between GUI events when executed in test cases and why certain events detect faults, so that they can better traverse the event space. New test adequacy criteria are defined for the EIG; new algorithms use these criteria and EIG to systematically generate test cases that are shown to be effective on four fielded open-source applications. %B ACM Transactions on Software Engineering and Methodology (TOSEM) %V 18 %P 7:1–7:35 - 7:1–7:35 %8 2008/11// %@ 1049-331X %G eng %U http://doi.acm.org/10.1145/1416563.1416567 %N 2 %R 10.1145/1416563.1416567 %0 Journal Article %J ACM Trans. Softw. Eng. Methodol. %D 2007 %T Designing and comparing automated test oracles for GUI-based software applications %A Xie,Qing %A Memon, Atif M. %K Graphical user interfaces %K GUI state %K GUI testing %K test oracles %K User interfaces %K widgets %X Test designers widely believe that the overall effectiveness and cost of software testing depends largely on the type and number of test cases executed on the software. This article shows that the test oracle, a mechanism that determines whether a software is executed correctly for a test case, also significantly impacts the fault detection effectiveness and cost of a test case. Graphical user interfaces (GUIs), which have become ubiquitous for interacting with today's software, have created new challenges for test oracle development. Test designers manually “assert” the expected values of specific properties of certain GUI widgets in each test case; during test execution, these assertions are used as test oracles to determine whether the GUI executed correctly. Since a test case for a GUI is a sequence of events, a test designer must decide: (1) what to assert; and (2) how frequently to check an assertion, for example, after each event in the test case or after the entire test case has completed execution. Variations of these two factors significantly impact the fault-detection ability and cost of a GUI test case. A technique to declaratively specify different types of automated GUI test oracles is described. Six instances of test oracles are developed and compared in an experiment on four software systems. The results show that test oracles do affect the fault detection ability of test cases in different and interesting ways: (1) Test cases significantly lose their fault detection ability when using “weak” test oracles; (2) in many cases, invoking a “thorough” oracle at the end of test case execution yields the best cost-benefit ratio; (3) certain test cases detect faults only if the oracle is invoked during a small “window of opportunity” during test execution; and (4) using thorough and frequently-executing test oracles can compensate for not having long test cases. %B ACM Trans. Softw. Eng. Methodol. %V 16 %8 2007/02// %@ 1049-331X %G eng %U http://doi.acm.org/10.1145/1189748.1189752 %N 1 %R 10.1145/1189748.1189752 %0 Journal Article %J Information VisualizationInformation Visualization %D 2007 %T Designing Semantic Substrates for Visual Network Exploration %A Aris,Aleks %A Shneiderman, Ben %K data exploration and analysis %K Graphical user interfaces %K Information Visualization %K network visualization %K semantic substrate design %X A semantic substrate is a spatial template for a network, where nodes are grouped into regions and laid out within each region according to one or more node attributes. This paper shows how users can be given control in designing their own substrates and how this ability leads to a different approach to network data exploration. Users can create a semantic substrate, enter their data, get feedback from domain experts, edit the semantic substrate, and iteratively continue this procedure until the domain experts are satisfied with the insights they have gained. We illustrate this process in two case studies with domain experts working with legal precedents and food webs. Guidelines for designing substrates are provided, including how to locate, size, and align regions in a substrate, which attributes to choose for grouping nodes into regions, how to select placement methods and which attributes to set as parameters of the selected placement method. Throughout the paper, examples are illustrated with NVSS 2.0, the network visualization tool developed to explore the semantic substrate idea. %B Information VisualizationInformation Visualization %V 6 %P 281 - 300 %8 2007/12/21/ %@ 1473-8716, 1473-8724 %G eng %U http://ivi.sagepub.com/content/6/4/281 %N 4 %R 10.1057/palgrave.ivs.9500162 %0 Journal Article %J Software Testing, Verification and Reliability %D 2007 %T An event-flow model of GUI-based applications for testing %A Memon, Atif M. %K event-driven software %K event-flow graph %K event-flow model %K Graphical user interfaces %K integration tree %K Model checking %K test oracles %K test-case generation %X Graphical user interfaces (GUIs) are by far the most popular means used to interact with today's software. The functional correctness of a GUI is required to ensure the safety, robustness and usability of an entire software system. GUI testing techniques used in practice are resource intensive; model-based automated techniques are rarely employed. A key reason for the reluctance in the adoption of model-based solutions proposed by researchers is their limited applicability; moreover, the models are expensive to create. Over the past few years, the present author has been developing different models for various aspects of GUI testing. This paper consolidates all of the models into one scalable event-flow model and outlines algorithms to semi-automatically reverse-engineer the model from an implementation. Earlier work on model-based test-case generation, test-oracle creation, coverage evaluation, and regression testing is recast in terms of this model by defining event-space exploration strategies (ESESs) and creating an end-to-end GUI testing process. Three such ESESs are described: for checking the event-flow model, test-case generation, and test-oracle creation. Two demonstrational scenarios show the application of the model and the three ESESs for experimentation and application in GUI testing. Copyright © 2007 John Wiley & Sons, Ltd. %B Software Testing, Verification and Reliability %V 17 %P 137 - 157 %8 2007/// %@ 1099-1689 %G eng %U http://onlinelibrary.wiley.com/doi/10.1002/stvr.364/abstract %N 3 %R 10.1002/stvr.364 %0 Conference Paper %B Information Visualization, 2007. IV '07. 11th International Conference %D 2007 %T Similarity-Based Forecasting with Simultaneous Previews: A River Plot Interface for Time Series Forecasting %A Buono,P. %A Plaisant, Catherine %A Simeone,A. %A Aris,A. %A Shneiderman, Ben %A Shmueli,G. %A Jank,W. %K data driven forecasting method %K data visualisation %K Data visualization %K Economic forecasting %K forecasting preview interface %K Graphical user interfaces %K historical time series dataset %K Laboratories %K new stock offerings %K partial time series %K pattern matching %K pattern matching search %K Predictive models %K river plot interface %K Rivers %K similarity-based forecasting %K Smoothing methods %K Technological innovation %K Testing %K time series %K time series forecasting %K Weather forecasting %X Time-series forecasting has a large number of applications. Users with a partial time series for auctions, new stock offerings, or industrial processes desire estimates of the future behavior. We present a data driven forecasting method and interface called similarity-based forecasting (SBF). A pattern matching search in an historical time series dataset produces a subset of curves similar to the partial time series. The forecast is displayed graphically as a river plot showing statistical information about the SBF subset. A forecasting preview interface allows users to interactively explore alternative pattern matching parameters and see multiple forecasts simultaneously. User testing with 8 users demonstrated advantages and led to improvements. %B Information Visualization, 2007. IV '07. 11th International Conference %I IEEE %P 191 - 196 %8 2007/07/04/6 %@ 0-7695-2900-3 %G eng %R 10.1109/IV.2007.101 %0 Conference Paper %B Software Engineering, 2007. ICSE 2007. 29th International Conference on %D 2007 %T Using GUI Run-Time State as Feedback to Generate Test Cases %A Xun Yuan %A Memon, Atif M. %K application under test %K automated test case generation %K Feedback %K feedback-based technique %K Graphical user interfaces %K GUI run-time state %K model-driven technique %K open-source software %K program testing %K public domain software %K reverse engineering %K reverse-engineering algorithm %K seed test suite %X This paper presents a new automated model-driven technique to generate test cases by using feedback from the execution of a "seed test suite" on an application under test (AUT). The test cases in the seed suite are designed to be generated automatically and executed very quickly. During their execution, feedback obtained from the AUT's run-time state is used to generate new, "improved" test cases. The new test cases subsequently become part of the seed suite. This "anytime technique" continues iteratively, generating and executing additional test cases until resources are exhausted or testing goals have been met. The feedback-based technique is demonstrated for automated testing of graphical user interfaces (GUIs). An existing abstract model of the GUI is used to automatically generate the seed test suite. It is executed; during its execution, state changes in the GUI pinpoint important relationships between GUI events, which evolve the model and help to generate new test cases. Together with a reverse- engineering algorithm used to obtain the initial model and seed suite, the feedback-based technique yields a fully automatic, end-to-end GUI testing process. A feasibility study on four large fielded open-source software (OSS) applications demonstrates that this process is able to significantly improve existing techniques and help identify/report serious problems in the OSS. In response, these problems have been fixed by the developers of the OSS in subsequent versions. %B Software Engineering, 2007. ICSE 2007. 29th International Conference on %P 396 - 405 %8 2007/05// %G eng %R 10.1109/ICSE.2007.94 %0 Journal Article %J IEEE Transactions on Visualization and Computer Graphics %D 2006 %T Balancing Systematic and Flexible Exploration of Social Networks %A Perer,A. %A Shneiderman, Ben %K Aggregates %K algorithms %K attribute ranking %K Cluster Analysis %K Computer Graphics %K Computer simulation %K Coordinate measuring machines %K coordinated views %K Data analysis %K data visualisation %K Data visualization %K exploratory data analysis %K Filters %K Gain measurement %K graph theory %K Graphical user interfaces %K Information Storage and Retrieval %K interactive graph visualization %K matrix algebra %K matrix overview %K Models, Biological %K Navigation %K network visualization %K Pattern analysis %K Population Dynamics %K Social Behavior %K social network analysis %K Social network services %K social networks %K social sciences computing %K Social Support %K SocialAction %K software %K statistical analysis %K statistical methods %K User-Computer Interface %X Social network analysis (SNA) has emerged as a powerful method for understanding the importance of relationships in networks. However, interactive exploration of networks is currently challenging because: (1) it is difficult to find patterns and comprehend the structure of networks with many nodes and links, and (2) current systems are often a medley of statistical methods and overwhelming visual output which leaves many analysts uncertain about how to explore in an orderly manner. This results in exploration that is largely opportunistic. Our contributions are techniques to help structural analysts understand social networks more effectively. We present SocialAction, a system that uses attribute ranking and coordinated views to help users systematically examine numerous SNA measures. Users can (1) flexibly iterate through visualizations of measures to gain an overview, filter nodes, and find outliers, (2) aggregate networks using link structure, find cohesive subgroups, and focus on communities of interest, and (3) untangle networks by viewing different link types separately, or find patterns across different link types using a matrix overview. For each operation, a stable node layout is maintained in the network visualization so users can make comparisons. SocialAction offers analysts a strategy beyond opportunism, as it provides systematic, yet flexible, techniques for exploring social networks %B IEEE Transactions on Visualization and Computer Graphics %V 12 %P 693 - 700 %8 2006/10//Sept %@ 1077-2626 %G eng %N 5 %R 10.1109/TVCG.2006.122 %0 Journal Article %J IEEE Transactions on Visualization and Computer Graphics %D 2006 %T Network Visualization by Semantic Substrates %A Shneiderman, Ben %A Aris,A. %K Automatic control %K data visualisation %K Data visualization %K Displays %K Filters %K Graphical user interfaces %K Information Visualization %K information visualization designers %K Law %K legal citations %K Legal factors %K legal precedent data %K network visualization %K NVSS 1.0 %K scalability %K semantic substrate %K Terminology %K Tunneling %K user-defined semantic substrates %X Networks have remained a challenge for information visualization designers because of the complex issues of node and link layout coupled with the rich set of tasks that users present. This paper offers a strategy based on two principles: (1) layouts are based on user-defined semantic substrates, which are non-overlapping regions in which node placement is based on node attributes, (2) users interactively adjust sliders to control link visibility to limit clutter and thus ensure comprehensibility of source and destination. Scalability is further facilitated by user control of which nodes are visible. We illustrate our semantic substrates approach as implemented in NVSS 1.0 with legal precedent data for up to 1122 court cases in three regions with 7645 legal citations %B IEEE Transactions on Visualization and Computer Graphics %V 12 %P 733 - 740 %8 2006/10//Sept %@ 1077-2626 %G eng %N 5 %R 10.1109/TVCG.2006.166 %0 Conference Paper %B Software Reliability Engineering, 2006. ISSRE '06. 17th International Symposium on %D 2006 %T Studying the Characteristics of a "Good" GUI Test Suite %A Xie,Qing %A Memon, Atif M. %K Fault detection %K Fault diagnosis %K graphical user interface testing %K Graphical user interfaces %K program debugging %K program testing %X The widespread deployment of graphical-user interfaces (GUIs) has increased the overall complexity of testing. A GUI test designer needs to perform the daunting task of adequately testing the GUI, which typically has very large input interaction spaces, while considering tradeoffs between GUI test suite characteristics such as the number of test cases (each modeled as a sequence of events), their lengths, and the event composition of each test case. There are no published empirical studies on GUI testing that a GUI test designer may reference to make decisions about these characteristics. Consequently, in practice, very few GUI testers know how to design their test suites. This paper takes the first step towards assisting in GUI test design by presenting an empirical study that evaluates the effect of these characteristics on testing cost and fault detection effectiveness. The results show that two factors significantly effect the fault-detection effectiveness of a test suite: (1) the diversity of states in which an event executes and (2) the event coverage of the suite. Test designers need to improve the diversity of states in which each event executes by developing a large number of short test cases to detect the majority of "shallow" faults, which are artifacts of modern GUI design. Additional resources should be used to develop a small number of long test cases to detect a small number of "deep" faults %B Software Reliability Engineering, 2006. ISSRE '06. 17th International Symposium on %P 159 - 168 %8 2006/11// %G eng %R 10.1109/ISSRE.2006.45 %0 Conference Paper %B Visual Analytics Science And Technology, 2006 IEEE Symposium On %D 2006 %T A Visual Interface for Multivariate Temporal Data: Finding Patterns of Events across Multiple Histories %A Fails,J. A %A Karlson,A. %A Shahamat,L. %A Shneiderman, Ben %K ball-and-chain visualization %K Chromium %K Computer science %K Data analysis %K data visualisation %K Data visualization %K Database languages %K event pattern discovery %K Graphical user interfaces %K History %K Information Visualization %K Medical treatment %K multivariate temporal data %K Pattern analysis %K pattern recognition %K PatternFinder integrated interface %K Query processing %K query visualization %K result-set visualization %K Spatial databases %K tabular visualization %K temporal pattern discovery %K temporal pattern searching %K Temporal query %K user interface %K User interfaces %K visual databases %K visual interface %X Finding patterns of events over time is important in searching patient histories, Web logs, news stories, and criminal activities. This paper presents PatternFinder, an integrated interface for query and result-set visualization for search and discovery of temporal patterns within multivariate and categorical data sets. We define temporal patterns as sequences of events with inter-event time spans. PatternFinder allows users to specify the attributes of events and time spans to produce powerful pattern queries that are difficult to express with other formalisms. We characterize the range of queries PatternFinder supports as users vary the specificity at which events and time spans are defined. Pattern Finder's query capabilities together with coupled ball-and-chain and tabular visualizations enable users to effectively query, explore and analyze event patterns both within and across data entities (e.g. patient histories, terrorist groups, Web logs, etc.) %B Visual Analytics Science And Technology, 2006 IEEE Symposium On %I IEEE %P 167 - 174 %8 2006/11/31/Oct. %@ 1-4244-0591-2 %G eng %R 10.1109/VAST.2006.261421 %0 Journal Article %J Journal of Software Maintenance and Evolution: Research and Practice %D 2005 %T Automating regression testing for evolving GUI software %A Memon, Atif M. %A Nagarajan,Adithya %A Xie,Qing %K daily/nightly builds %K event-flow graphs %K Graphical user interfaces %K GUI regression testing %K GUI testing %K smoke testing %K Software quality %X With the widespread deployment of broadband connections worldwide, software development and maintenance are increasingly being performed by multiple engineers, often working around-the-clock to maximize code churn rates. To ensure rapid quality assurance of such software, techniques such as ‘nightly/daily building and smoke testing’ have become widespread since they often reveal bugs early in the software development process. During these builds, a development version of the software is checked out from the source code repository tree, compiled, linked, and (re)tested with the goal of (re)validating its basic functionality. Although successful for conventional software, smoke tests are difficult to develop and automatically re-run for software that has a graphical user interface (GUI). In this paper, we describe a framework called DART (Daily Automated Regression Tester) that addresses the needs of frequent and automated re-testing of GUI software. The key to our success is automation: DART automates everything from structural GUI analysis, smoke-test-case generation, test-oracle creation, to code instrumentation, test execution, coverage evaluation, regeneration of test cases, and their re-execution. Together with the operating system's task scheduler, DART can execute frequently with little input from the developer/tester to re-test the GUI software. We provide results of experiments showing the time taken and memory required for GUI analysis, test case and test oracle generation, and test execution. We empirically compare the relative costs of employing different levels of detail in the GUI test oracle. We also show the events and statements covered by the smoke test cases. Copyright © 2005 John Wiley & Sons, Ltd. %B Journal of Software Maintenance and Evolution: Research and Practice %V 17 %P 27 - 64 %8 2005/// %@ 1532-0618 %G eng %U http://onlinelibrary.wiley.com/doi/10.1002/smr.305/abstract %N 1 %R 10.1002/smr.305 %0 Conference Paper %B Software Maintenance, 2005. ICSM'05. Proceedings of the 21st IEEE International Conference on %D 2005 %T Rapid "crash testing" for continuously evolving GUI-based software applications %A Xie,Q. %A Memon, Atif M. %K crash testing %K graphical user interface software retesting %K Graphical user interfaces %K GUI-based software application %K immediate feedback %K program testing %K rapid-feedback-based quality assurance %K software evolution %K Software maintenance %K software prototyping %K Software quality %X Several rapid-feedback-based quality assurance mechanisms are used to manage the quality of continuously evolving software. Even though graphical user interfaces (GUIs) are one of the most important parts of software, there are currently no mechanisms to quickly retest evolving GUI software. We leverage our previous work on GUI testing to define a new automatic GUI re-testing process called "crash testing" that is integrated with GUI evolution. We describe two levels of crash testing: (1) immediate feedback-based in which a developer indicates that a GUI bug was fixed in response to a previously reported crash; only select crash test cases are rerun and the developer is notified of the results in a matter of seconds, and (2) between code changes in which new crash test cases are generated on-the-fly and executed on the GUI. Since the code may be changed by another developer before all the crash tests have been executed, hence requiring restarting of the process, we use a simple rotation-based scheme to ensure that all crash tests are executed over a series of code changes. We show, via empirical studies, that our crash tests are effective at revealing serious problems in the GUI. %B Software Maintenance, 2005. ICSM'05. Proceedings of the 21st IEEE International Conference on %P 473 - 482 %8 2005/09// %G eng %R 10.1109/ICSM.2005.72 %0 Conference Paper %B 2005 IEEE Symposium on Visual Languages and Human-Centric Computing %D 2005 %T Show Me! Guidelines for producing recorded demonstrations %A Plaisant, Catherine %A Shneiderman, Ben %K animation %K auditory cues %K Computer aided instruction %K Computer animation %K Computer Graphics %K Computer interfaces %K computer literacy %K content guidelines %K documentation %K Government %K Graphical user interfaces %K Guidelines %K Laboratories %K narration %K recorded demonstrations %K screen capture animation %K technical guidelines %K usability %K User interfaces %K visual appeal %K visual cues %X Although recorded demonstrations (screen capture animations with narration) have become a popular form of instruction for user interfaces, little work has been done to describe guidelines for their design. Based on our experience in several projects, we offer a starting set of guidelines for the design of visually appealing and cognitively effective recorded demonstrations. Technical guidelines encourage users to keep file sizes small, strive for universal usability, and ensure user control etc. and provide tips to achieve those goals. Content guidelines include: create short demonstrations that focus on tasks, highlight each step with auditory and visual cues, synchronize narration and animation carefully, and create demonstrations with a clear beginning, middle, and end. %B 2005 IEEE Symposium on Visual Languages and Human-Centric Computing %I IEEE %P 171 - 178 %8 2005/09/20/24 %@ 0-7695-2443-5 %G eng %R 10.1109/VLHCC.2005.57 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2005 %T Studying the fault-detection effectiveness of GUI test cases for rapidly evolving software %A Memon, Atif M. %A Xie,Q. %K daily automated regression tester %K Fault diagnosis %K fault-detection %K formal specification %K formal verification %K Graphical user interfaces %K GUI test cases %K program testing %K quality assurance mechanism %K rapidly evolving software %K smoke regression testing technique %K software development %K software fault tolerance %K Software maintenance %K software prototyping %K Software quality %K test oracles %X Software is increasingly being developed/maintained by multiple, often geographically distributed developers working concurrently. Consequently, rapid-feedback-based quality assurance mechanisms such as daily builds and smoke regression tests, which help to detect and eliminate defects early during software development and maintenance, have become important. This paper addresses a major weakness of current smoke regression testing techniques, i.e., their inability to automatically (re)test graphical user interfaces (GUIs). Several contributions are made to the area of GUI smoke testing. First, the requirements for GUI smoke testing are identified and a GUI smoke test is formally defined as a specialized sequence of events. Second, a GUI smoke regression testing process called daily automated regression tester (DART) that automates GUI smoke testing is presented. Third, the interplay between several characteristics of GUI smoke test suites including their size, fault detection ability, and test oracles is empirically studied. The results show that: 1) the entire smoke testing process is feasible in terms of execution time, storage space, and manual effort, 2) smoke tests cannot cover certain parts of the application code, 3) having comprehensive test oracles may make up for not having long smoke test cases, and 4) using certain oracles can make up for not having large smoke test suites. %B Software Engineering, IEEE Transactions on %V 31 %P 884 - 896 %8 2005/10// %@ 0098-5589 %G eng %N 10 %R 10.1109/TSE.2005.117 %0 Journal Article %J Information VisualizationInformation Visualization %D 2004 %T Dynamic Query Tools for Time Series Data Sets: Timebox Widgets for Interactive Exploration %A Hochheiser,Harry %A Shneiderman, Ben %K angular queries %K Bioinformatics %K dynamic query %K Graphical user interfaces %K temporal data %K time series %K timeboxes %K TimeSearcher %K visual query %X Timeboxes are rectangular widgets that can be used in direct-manipulation graphical user interfaces (GUIs) to specify query constraints on time series data sets. Timeboxes are used to specify simultaneously two sets of constraints: given a set of N time series profiles, a timebox covering time periods x1…x2 (x1 ≤ x2) and values y1…y2 (y1 ≤ y2) will retrieve only those n√N that have values y1 ≤ y2 during all times x1 ≤ x ≤ x2. TimeSearcher is an information visualization tool that combines timebox queries with overview displays, query-by-example facilities, and support for queries over multiple time-varying attributes. Query manipulation tools including pattern inversion and ‘leaders & laggards’ graphical bookmarks provide additional support for interactive exploration of data sets. Extensions to the basic timebox model that provide additional expressivity include variable time timeboxes, which can be used to express queries with variability in the time interval, and angular queries, which search for ranges of differentials, rather than absolute values. Analysis of the algorithmic requirements for providing dynamic query performance for timebox queries showed that a sequential search outperformed searches based on geometric indices. Design studies helped identify the strengths and weaknesses of the query tools. Extended case studies involving the analysis of two different types of data from molecular biology experiments provided valuable feedback and validated the utility of both the timebox model and the TimeSearcher tool. Timesearcher is available at http://www.cs.umd.edu/hcil/timesearcher %B Information VisualizationInformation Visualization %V 3 %P 1 - 18 %8 2004/03/20/ %@ 1473-8716, 1473-8724 %G eng %U http://ivi.sagepub.com/content/3/1/1 %N 1 %R 10.1057/palgrave.ivs.9500061 %0 Conference Paper %B Software Maintenance, 2004. Proceedings. 20th IEEE International Conference on %D 2004 %T Empirical evaluation of the fault-detection effectiveness of smoke regression test cases for GUI-based software %A Memon, Atif M. %A Xie,Qing %K daily automated regression tester %K daily builds %K fault-detection effectiveness %K graphical user interface %K Graphical user interfaces %K GUI-based software %K program testing %K Quality assurance %K Regression analysis %K smoke regression test cases %K software development %K software fault tolerance %K Software maintenance %K Software quality %K software quality assurance %K test oracle complexity %K test oracles %K test-case length %X Daily builds and smoke regression tests have become popular quality assurance mechanisms to detect defects early during software development and maintenance. In previous work, we addressed a major weakness of current smoke regression testing techniques, i.e., their lack of ability to automatically (re)test graphical user interface (GUI) event interactions - we presented a GUI smoke regression testing process called daily automated regression tester (DART). We have deployed DART and have found several interesting characteristics of GUI smoke tests that we empirically demonstrate in this paper. We also combine smoke tests with different types of test oracles and present guidelines for practitioners to help them generate and execute the most effective combinations of test-case length and test oracle complexity. Our experimental subjects consist of four GUI-based applications. We generate 5000-8000 smoke tests (enough to be run in one night) for each application. Our results show that: (1) short GUI smoke tests with certain test oracles are effective at detecting a large number of faults; (2) there are classes of faults that our smoke test cannot detect; (3) short smoke tests execute a large percentage of code; and (4) the entire smoke testing process is feasible to do in terms of execution time and storage space. %B Software Maintenance, 2004. Proceedings. 20th IEEE International Conference on %P 8 - 17 %8 2004/09// %G eng %R 10.1109/ICSM.2004.1357785 %0 Conference Paper %B Eighth International Conference on Information Visualisation, 2004. IV 2004. Proceedings %D 2004 %T Extending the utility of treemaps with flexible hierarchy %A Chintalapani,G. %A Plaisant, Catherine %A Shneiderman, Ben %K 2D displays %K Computer displays %K Computer science %K data visualisation %K Data visualization %K Educational institutions %K flexible hierarchy %K graphical user interface %K Graphical user interfaces %K hierarchical information %K Nominations and elections %K Switches %K Tree data structures %K Tree graphs %K Treemap 4.0 %K Two dimensional displays %K usability %K visualization technique %X Treemaps are a visualization technique for presenting hierarchical information on two-dimensional displays. Prior implementations limit the visualization to pre-defined static hierarchies. Flexible hierarchy, a new capability of Treemap 4.0, enables users to define various hierarchies through dynamically selecting a series of data attributes so that they can discover patterns, clusters and outliers. This work describes the design and implementation issues of flexible hierarchy. It then reports on a usability study, which led to enhancements to the interface. %B Eighth International Conference on Information Visualisation, 2004. IV 2004. Proceedings %I IEEE %P 335 - 344 %8 2004/07/14/16 %@ 0-7695-2177-0 %G eng %R 10.1109/IV.2004.1320166 %0 Conference Paper %B Automated Software Engineering, 2004. Proceedings. 19th International Conference on %D 2004 %T Using transient/persistent errors to develop automated test oracles for event-driven software %A Memon, Atif M. %A Xie,Qing %K automated test oracles %K Automatic testing %K event driven software %K Graphical user interfaces %K persistent errors %K program testing %K resource allocation %K resource utilization %K software intensive systems %K test case execution %K transient errors %X Today's software-intensive systems contain an important class of software, namely event-driven software (EDS). All EDS take events as input, change their state, and (perhaps) output an event sequence. EDS is typically implemented as a collection of event-handlers designed to respond to individual events. The nature of EDS creates new challenges for test automation. In this paper, we focus on those relevant to automated test oracles. A test oracle is a mechanism that determines whether a software executed correctly for a test case. A test case for an EDS consists of a sequence of events. The test case is executed on the EDS, one event at a time. Errors in the EDS may "appear" and later ”disappear" at several points (e.g., after an event is executed) during test case execution. Because of the behavior of these transient (those that disappear) and persistent (those that don't disappear) errors, EDS require complex and expensive test oracles that compare the expected and actual output multiple times during test case execution. We leverage our previous work to study several applications and observe the occurrence of persistent/transient errors. Our studies show that in practice, a large number of errors in EDS are transient and that there are specific classes of events that lead to transient errors. We use the results of this study to develop a new test oracle that compares the expected and actual output at strategic points during test case execution. We show that the oracle is effective at detecting errors and efficient in terms of resource utilization %B Automated Software Engineering, 2004. Proceedings. 19th International Conference on %P 186 - 195 %8 2004/09// %G eng %R 10.1109/ASE.2004.1342736 %0 Conference Paper %B Software Maintenance, 2003. ICSM 2003. Proceedings. International Conference on %D 2003 %T DART: a framework for regression testing "nightly/daily builds" of GUI applications %A Memon, Atif M. %A Banerjee,I. %A Hashmi,N. %A Nagarajan,A. %K automated retesting %K automatic test software %K coverage evaluation %K daily automated regression tester %K DART %K frequent retesting %K graphical user interface %K Graphical user interfaces %K GUI software %K instrumentation coding %K program testing %K regression testing %K Software development management %K Software development process %K Software maintenance %K Software quality %K structural GUI analysis %K Test Case Generation %K test cases regeneration %K Test execution %K test oracle creation %X "Nightly/daily building and smoke testing" have become widespread since they often reveal bugs early in the software development process. During these builds, software is compiled, linked, and (re)tested with the goal of validating its basic functionality. Although successful for conventional software, smoke tests are difficult to develop and automatically rerun for software that has a graphical user interface (GUI). In this paper, we describe a framework called DART (daily automated regression tester) that addresses the needs of frequent and automated re-testing of GUI software. The key to our success is automation: DART automates everything from structural GUI analysis; test case generation; test oracle creation; to code instrumentation; test execution; coverage evaluation; regeneration of test cases; and their re-execution. Together with the operating system's task scheduler, DART can execute frequently with little input from the developer/tester to retest the GUI software. We provide results of experiments showing the time taken and memory required for GUI analysis, test case and test oracle generation, and test execution. We also empirically compare the relative costs of employing different levels of detail in the GUI test cases. %B Software Maintenance, 2003. ICSM 2003. Proceedings. International Conference on %P 410 - 419 %8 2003/09// %G eng %R 10.1109/ICSM.2003.1235451 %0 Conference Paper %B Automated Software Engineering, 2003. Proceedings. 18th IEEE International Conference on %D 2003 %T What test oracle should I use for effective GUI testing? %A Memon, Atif M. %A Banerjee,I. %A Nagarajan,A. %K empirical studies %K graphical user interface %K Graphical user interfaces %K GUI testing %K oracle information %K oracle procedure %K oracle space requirement %K oracle time requirements %K program testing %K software engineering %K Software testing %K test cost %K test effectiveness %K test oracle %X Test designers widely believe that the overall effectiveness and cost of software testing depends largely on the type and number of test cases executed on the software. In this paper we show that the test oracle used during testing also contributes significantly to test effectiveness and cost. A test oracle is a mechanism that determines whether software executed correctly for a test case. We define a test oracle to contain two essential parts: oracle information that represents expected output; and an oracle procedure that compares the oracle information with the actual output. By varying the level of detail of oracle information and changing the oracle procedure, a test designer can create different types of test oracles. We design 11 types of test oracles and empirically compare them on four software systems. We seed faults in software to create 100 faulty versions, execute 600 test cases on each version, for all 11 types of oracles. In all, we report results of 660,000 test runs on software. We show (1) the time and space requirements of the oracles, (2) that faults are detected early in the testing process when using detailed oracle information and complex oracle procedures, although at a higher cost per test case, and (3) that employing expensive oracles results in detecting a large number of faults using relatively smaller number of test cases. %B Automated Software Engineering, 2003. Proceedings. 18th IEEE International Conference on %P 164 - 173 %8 2003/10// %G eng %R 10.1109/ASE.2003.1240304 %0 Journal Article %J ACM SIGCAPH Computers and the Physically Handicapped %D 2002 %T Promoting universal usability with multi-layer interface design %A Shneiderman, Ben %K first-time user %K Graphical user interfaces %K multi-layer interface %K novice user %K online help %K universal usability %X Increased interest in universal usability is causing some researchers to study advanced strategies for satisfying first-time as well as intermittent and expert users. This paper promotes the idea of multi-layer interface designs that enable first-time and novice users to begin with a limited set of features at layer 1. They can remain at layer 1, then move up to higher layers when needed or when they have time to learn further features. The arguments for and against multi-layer interfaces are presented with two example systems: a word processor with 8 layers and an interactive map with 3 layers. New research methods and directions are proposed. %B ACM SIGCAPH Computers and the Physically Handicapped %P 1 - 8 %8 2002/06// %@ 0163-5727 %G eng %U http://doi.acm.org/10.1145/960201.957206 %N 73-74 %R 10.1145/960201.957206 %0 Journal Article %J Software Engineering, IEEE Transactions on %D 2001 %T Hierarchical GUI test case generation using automated planning %A Memon, Atif M. %A Pollack,M. E %A Soffa,M. L %K Artificial intelligence %K automated planning %K automatic test case generation %K Automatic testing %K correctness testing %K goal state %K Graphical user interfaces %K hierarchical GUI test case generation %K initial state %K Microsoft WordPad %K operators %K plan-generation system %K planning (artificial intelligence) %K Planning Assisted Tester for Graphical User Interface Systems %K program testing %K software %X The widespread use of GUIs for interacting with software is leading to the construction of more and more complex GUIs. With the growing complexity come challenges in testing the correctness of a GUI and its underlying software. We present a new technique to automatically generate test cases for GUIs that exploits planning, a well-developed and used technique in artificial intelligence. Given a set of operators, an initial state, and a goal state, a planner produces a sequence of the operators that will transform the initial state to the goal state. Our test case generation technique enables efficient application of planning by first creating a hierarchical model of a GUI based on its structure. The GUI model consists of hierarchical planning operators representing the possible events in the GUI. The test designer defines the preconditions and effects of the hierarchical operators, which are input into a plan-generation system. The test designer also creates scenarios that represent typical initial and goal states for a GUI user. The planner then generates plans representing sequences of GUI interactions that a user might employ to reach the goal state from the initial state. We implemented our test case generation system, called Planning Assisted Tester for Graphical User Interface Systems (PATHS) and experimentally evaluated its practicality and effectiveness. We describe a prototype implementation of PATHS and report on the results of controlled experiments to generate test cases for Microsoft's WordPad %B Software Engineering, IEEE Transactions on %V 27 %P 144 - 155 %8 2001/02// %@ 0098-5589 %G eng %N 2 %R 10.1109/32.908959 %0 Journal Article %J ACM Trans. Comput.-Hum. Interact. %D 2000 %T Creating creativity: user interfaces for supporting innovation %A Shneiderman, Ben %K creativity support tools %K direct manipulation %K Graphical user interfaces %K Human-computer interaction %K Information Visualization %X A challenge for human-computer interaction researchers and user interf ace designers is to construct information technologies that support creativity. This ambitious goal can be attained by building on an adequate understanding of creative processes. This article offers a four-phase framework for creativity that might assist designers in providing effective tools for users: (1)Collect: learn from provious works stored in libraries, the Web, etc.; (2) Relate: consult with peers and mentors at early, middle, and late stages, (3)Create: explore, compose, evaluate possible solutions; and (4) Donate: disseminate the results and contribute to the libraries. Within this integrated framework, this article proposes eight activities that require human-computer interaction research and advanced user interface design. A scenario about an architect illustrates the process of creative work within such an environment. %B ACM Trans. Comput.-Hum. Interact. %V 7 %P 114 - 138 %8 2000/03// %@ 1073-0516 %G eng %U http://doi.acm.org/10.1145/344949.345077 %N 1 %R 10.1145/344949.345077 %0 Conference Paper %B IEEE International Conference on Information Visualization, 2000. Proceedings %D 2000 %T Direct annotation: a drag-and-drop strategy for labeling photos %A Shneiderman, Ben %A Kang,H. %K Biomedical imaging %K Cities and towns %K Computer errors %K Computer science %K database indexing %K database schema %K Databases %K digital libraries %K direct annotation %K direct manipulation %K drag-and-drop strategy %K Educational institutions %K graphical user interface %K Graphical user interfaces %K History %K hobby computing %K image searching %K label placement %K Labeling %K Laboratories %K Libraries %K personal information systems %K personal names database %K PhotoFinder prototype %K photograph labelling %K photographic libraries %K Photography %K scrolling list %K user interface design %K visual databases %X Annotating photographs is such a time-consuming, tedious and error-prone data entry task that it discourages most owners of personal photo libraries. By allowing the user to drag labels, such as personal names, from a scrolling list and drop them onto a photo, we believe we can make the task faster, easier and more appealing. Since the names are entered in a database, searching for all photos of a friend or family member is dramatically simplified. We describe the user interface design and the database schema to support direct annotation, as implemented in our PhotoFinder prototype %B IEEE International Conference on Information Visualization, 2000. Proceedings %I IEEE %P 88 - 95 %8 2000/// %@ 0-7695-0743-3 %G eng %R 10.1109/IV.2000.859742 %0 Conference Paper %D 2000 %T Loki: a state-driven fault injector for distributed systems %A Chandra,R. %A Lefever,R.M. %A Michel Cukier %A Sanders,W. H. %K campaign execution %K computer testing %K distributed applications failure %K distributed processing %K distributed system validation %K Fault diagnosis %K fault event placement %K fault specification %K global system state %K global timeline %K Graphical user interfaces %K Loki %K off-line clock synchronization %K post-runtime analysis %K specified measures estimation %K state machine specification %K state-driven fault injection %K Synchronisation %X Distributed applications can fail in subtle ways that depend on the state of multiple parts of a system. This complicates the validation of such systems via fault injection, since it suggests that faults should be injected based on the global state of the system. In Loki, fault injection is performed based on a partial view of the global state of a distributed system, i.e. faults injected in one node of the system can depend on the state of other nodes. Once faults are injected, a post-runtime analysis, using off-line clock synchronization, is used to place events and injections on a single global timeline and to determine whether the intended faults were properly injected. Finally, experiments containing successful fault injections are used to estimate the specified measures. In addition to briefly reviewing the concepts behind Loki and its organization, we detail Loki's user interface. In particular, we describe the graphical user interfaces for specifying state machines and faults, for executing a campaign and for verifying whether the faults were properly injected %P 237 - 242 %8 2000/// %G eng %R 10.1109/ICDSN.2000.857544 %0 Conference Paper %B Proceedings of the fifth ACM conference on Digital libraries %D 2000 %T Visualizing digital library search results with categorical and hierarchical axes %A Shneiderman, Ben %A Feldman,David %A Rose,Anne %A Grau,Xavier Ferré %K categorical axes %K digital libraries %K Graphical user interfaces %K hierarchy %K hieraxes %K Information Visualization %X Digital library search results are usually shown as a textual list, with 10-20 items per page. Viewing several thousand search results at once on a two-dimensional display with continuous variables is a promising alternative. Since these displays can overwhelm some users, we created a simplified two-dimensional display that uses categorical and hierarchical axes, called hieraxes. Users appreciate the meaningful and limited number of terms on each hieraxis. At each grid point of the display we show a cluster of color-coded dots or a bar chart. Users see the entire result set and can then click on labels to move down a level in the hierarchy. Handling broad hierarchies and arranging for imposed hierarchies led to additional design innovations. We applied hieraxes to a digital video library of science topics used by middle school teachers, a legal information system, and a technical library using the ACM Computing Classification System. Feedback from usability testing with 32 subjects revealed strengths and weaknesses. %B Proceedings of the fifth ACM conference on Digital libraries %S DL '00 %I ACM %C New York, NY, USA %P 57 - 66 %8 2000/// %@ 1-58113-231-X %G eng %U http://doi.acm.org/10.1145/336597.336637 %R 10.1145/336597.336637 %0 Conference Paper %B Software Engineering, 1999. Proceedings of the 1999 International Conference on %D 1999 %T Using a goal-driven approach to generate test cases for GUIs %A Memon, Atif M. %A Pollack,M. E %A Soffa,M. L %K Artificial intelligence %K automatic test case generation %K goal state %K goal-driven approach %K Graphical user interfaces %K GUIs %K hierarchical planning operators %K initial state %K Microsoft Word-Pad %K operators %K planning (artificial intelligence) %K program testing %K software %K verification commands %X The widespread use of GUIs for interacting with software is leading to the construction of more and more complex GUIs. With the growing complexity comes challenges in testing the correctness of a GUI and the underlying software. We present a new technique to automatically generate test cases for GUIs that exploits planning, a well developed and used technique in artificial intelligence. Given a set of operators, an initial state and a goal state, a planner produces a sequence of the operators that will change the initial state to the goal state. Our test case generation technique first analyzes a GUI and derives hierarchical planning operators from the actions in the GUI. The test designer determines the preconditions and effects of the hierarchical operators, which are then input into a planning system. With the knowledge of the GUI and the way in which the user will interact with the GUI, the test designer creates sets of initial and goal states. Given these initial and final states of the GUI, a hierarchical planner produces plans, or a set of test cases, that enable the goal state to be reached. Our technique has the additional benefit of putting verification commands into the test cases automatically. We implemented our technique by developing the GUI analyzer and extending a planner. We generated test cases for Microsoft's Word-Pad to demonstrate the viability and practicality of the approach. %B Software Engineering, 1999. Proceedings of the 1999 International Conference on %P 257 - 266 %8 1999/05// %G eng %0 Journal Article %J Information Processing & Management %D 1998 %T An information architecture to support the visualization of personal histories %A Plaisant, Catherine %A Shneiderman, Ben %A Mushlin,Rich %K Graphical user interfaces %K Information Visualization %K LifeLines %K medical patient record %K personal histories %K temporal data %K timelines %X This paper proposes an information architecture for personal history data and describes how the data model can be extended to a runtime model for a compact visualization using graphical timelines. Our information architecture was developed for juvenile justice and medical patient records, but is usable in other application domains such as personal resumes, financial histories, or customer support. Our model groups personal history events into aggregates that are contained in facets (e.g., doctor visits, hospitalizations, or lab tests). Crosslinks enable representation of arbitrary relationships across events and aggregates. Data attributes, such as severity, can be mapped by data administrators to visual attributes such as color and line thickness. End-users have powerful controls over the display contents, and they can modify the mapping to fit their tasks. %B Information Processing & Management %V 34 %P 581 - 597 %8 1998/09// %@ 0306-4573 %G eng %U http://www.sciencedirect.com/science/article/pii/S0306457398000247 %N 5 %R 10.1016/S0306-4573(98)00024-7 %0 Conference Paper %B IEEE Symposium on Information Visualization, 1997. Proceedings %D 1997 %T Design and evaluation of incremental data structures and algorithms for dynamic query interfaces %A Tanin,E. %A Beigel,R. %A Shneiderman, Ben %K Algorithm design and analysis %K Bars %K Computer science %K continuous real-time feedback %K Data structures %K data visualisation %K Data visualization %K database access mechanism %K Displays %K DQI algorithms %K dynamic query interfaces %K Feedback %K Graphical user interfaces %K Heuristic algorithms %K incremental data structures %K Information Visualization %K large databases %K Manipulator dynamics %K NASA %K query formulation %K query languages %K Query processing %K real-time systems %K small databases %K User interfaces %K very large databases %K visual databases %K visual languages %X A dynamic query interface (DQI) is a database access mechanism that provides continuous real-time feedback to the user during query formulation. Previous work shows that DQIs are elegant and powerful interfaces to small databases. Unfortunately, when applied to large databases, previous DQI algorithms slow to a crawl. We present a new incremental approach to DQI algorithms and display updates that work well with large databases, both in theory and in practice. %B IEEE Symposium on Information Visualization, 1997. Proceedings %I IEEE %P 81 - 86 %8 1997/10/21/21 %@ 0-8186-8189-6 %G eng %R 10.1109/INFVIS.1997.636790 %0 Journal Article %J IEEE Transactions on Software Engineering %D 1997 %T Visual and textual consistency checking tools for graphical user interfaces %A Mahajan,R. %A Shneiderman, Ben %K button analysis tools %K Color %K dialog box summary table %K experiment %K graphical analysis tools %K Graphical user interfaces %K human factors %K inconsistent interface terminology %K interface color %K interface spell checker %K Output feedback %K Prototypes %K SHERLOCK %K Software architecture %K Software design %K software metrics %K software prototyping %K software tools %K Terminology %K terminology analysis tools %K Testing %K textual consistency checking tools %K user interface management systems %K User interfaces %K user performance %K visual consistency checking tools %X Designing user interfaces with consistent visual and textual properties is difficult. To demonstrate the harmful effects of inconsistency, we conducted an experiment with 60 subjects. Inconsistent interface terminology slowed user performance by 10 to 25 percent. Unfortunately, contemporary software tools provide only modest support for consistency control. Therefore, we developed SHERLOCK, a family of consistency analysis tools, which evaluates visual and textual properties of user interfaces. It provides graphical analysis tools such as a dialog box summary table that presents a compact overview of visual properties of all dialog boxes. SHERLOCK provides terminology analysis tools including an interface concordance, an interface spellchecker, and terminology baskets to check for inconsistent use of familiar groups of terms. Button analysis tools include a button concordance and a button layout table to detect variant capitalization, distinct typefaces, distinct colors, variant button sizes, and inconsistent button placements. We describe the design, software architecture, and the use of SHERLOCK. We tested SHERLOCK with four commercial prototypes. The outputs, analysis, and feedback from designers of the applications are presented %B IEEE Transactions on Software Engineering %V 23 %P 722 - 735 %8 1997/11// %@ 0098-5589 %G eng %N 11 %R 10.1109/32.637386 %0 Conference Paper %B , IEEE Symposium on Visual Languages, 1996. Proceedings %D 1996 %T The eyes have it: a task by data type taxonomy for information visualizations %A Shneiderman, Ben %K advanced graphical user interface design %K Art %K data mining %K data type taxonomy %K data visualisation %K Data visualization %K Displays %K Eyes %K Graphical user interfaces %K Information filtering %K Information filters %K information visualizations %K multi dimensional data %K Multimedia databases %K network data %K Taxonomy %K visual databases %K visual information seeking %K visual programming %X A useful starting point for designing advanced graphical user interfaces is the visual information seeking Mantra: overview first, zoom and filter, then details on demand. But this is only a starting point in trying to understand the rich and varied set of information visualizations that have been proposed in recent years. The paper offers a task by data type taxonomy with seven data types (one, two, three dimensional data, temporal and multi dimensional data, and tree and network data) and seven tasks (overview, zoom, filter, details-on-demand, relate, history, and extracts) %B , IEEE Symposium on Visual Languages, 1996. Proceedings %I IEEE %P 336 - 343 %8 1996/09/03/6 %@ 0-8186-7508-X %G eng %R 10.1109/VL.1996.545307 %0 Journal Article %J IEEE Software %D 1995 %T Image-browser taxonomy and guidelines for designers %A Plaisant, Catherine %A Carr,D. %A Shneiderman, Ben %K analysis %K Computer Graphics %K design %K designer guidelines %K Equations %K Europe %K Evaluation %K Formal specifications %K Graphical user interfaces %K Guidelines %K IMAGE PROCESSING %K image-browser taxonomy %K informal specification technique %K Laboratories %K large image browsing %K Layout %K Road transportation %K selected image exploration %K SHAPE %K Software design %K task taxonomy %K Taxonomy %K tools %K two-dimensional browsing %K user interface management systems %K visual databases %X In many applications users must browse large images. Most designers merely use two one-dimensional scroll bars or ad hoc designs for two-dimensional scroll bars. However, the complexity of two-dimensional browsing suggests that more careful analysis, design, and evaluation might lead to significant improvements. Our exploration of existing 2D browsers has led us to identify many features and a wide variety of tasks performed with the browsers. We introduce an informal specification technique to describe 2D browsers and a task taxonomy, suggest design features and guidelines, and assess existing strategies. We focus on the tools to explore a selected image and so do not cover techniques to browse a series of images or to browse large-image databases %B IEEE Software %V 12 %P 21 - 32 %8 1995/03// %@ 0740-7459 %G eng %N 2 %R 10.1109/52.368260 %0 Conference Paper %B Proceedings of the Fourth Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises, 1995 %D 1995 %T Organization overviews and role management: inspiration for future desktop environments %A Plaisant, Catherine %A Shneiderman, Ben %K Asia %K bank data processing %K Databases %K Environmental economics %K Environmental management %K future desktop environments %K Graphical user interfaces %K human resource management %K management information systems %K Management training %K multiple personal roles %K office automation %K organization overviews %K personnel %K Project management %K Prototypes %K role management %K role-centered approach %K scheduling %K semi-automated searches %K User interfaces %K Utility programs %K World Bank %X In our exploration of future work environments for the World Bank we proposed two concepts. First, organization overviews provide a consistent support to present the results of a variety of manual or semi-automated searches. Second this view can be adapted or expanded for each class of users to finally map the multiple personal roles an individual has in an organization. After command line interfaces, graphical user interfaces, and the current “docu-centric” designs, a natural direction is towards a role-centered approach where we believe the emphasis is on the management of those multiple roles. Large visual overviews of the organization can be rapidly manipulated and zoomed in on to reveal the multiple roles each individual plays. Each role involves coordination with groups of people and accomplishment of tasks within a schedule %B Proceedings of the Fourth Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises, 1995 %I IEEE %P 14 - 22 %8 1995/04/20/22 %@ 0-8186-7019-3 %G eng %R 10.1109/ENABL.1995.484544 %0 Journal Article %J IEEE Software %D 1994 %T Dynamic queries for visual information seeking %A Shneiderman, Ben %K Algorithm design and analysis %K animated results %K animation %K Application software %K Command languages %K complex queries %K database management systems %K Databases %K display algorithms %K Displays %K dynamic queries %K Educational institutions %K Games %K Graphical user interfaces %K human factors %K Query processing %K retrieval %K Runtime %K Technological innovation %K user-interface design %K visual databases %K visual information seeking %K visual interfaces %K widgets %X Considers how dynamic queries allow users to "fly through" databases by adjusting widgets and viewing the animated results. In studies, users reacted to this approach with an enthusiasm more commonly associated with video games. Adoption requires research into retrieval and display algorithms and user-interface design. The author discusses how experts may benefit from visual interfaces because they will be able to formulate more complex queries and interpret intricate results.<> %B IEEE Software %V 11 %P 70 - 77 %8 1994/11// %@ 0740-7459 %G eng %N 6 %R 10.1109/52.329404 %0 Journal Article %J Information Systems %D 1993 %T A graphical query interface based on aggregation/generalization hierarchies %A Weiland,William J %A Shneiderman, Ben %K aggregation/generalization hierarchies %K empirical test %K Graphical user interfaces %K visual queries %X In order for automated information systems to be used effectively, they must be made easily accessible to a wide range of users and with short training periods. This work proposes a method of organizing documents based on the concepts of aggregation and generalization hierarchies. We propose a graphical user interface to provide a more intuitive form of Boolean query. This design is based on mapping the nodes of the aggregation hierarchy to Boolean intersection operations, mapping the nodes of the generalization hierarchy to Boolean union operations, and providing a concrete, graphical, manipulable representation of both of these node types. Finally, a working prototype interface was constructed and evaluated experimentally against a classical command-line Boolean query interface. In this formative evaluation with sixteen subjects, the graphical interface produced less than one-tenth the errors of the textual interface, on average. Significant differences in time spent specifying queries were not found. Observations and comments provide guidance for designers. %B Information Systems %V 18 %P 215 - 232 %8 1993/06// %@ 0306-4379 %G eng %U http://www.sciencedirect.com/science/article/pii/030643799390018V %N 4 %R 10.1016/0306-4379(93)90018-V %0 Journal Article %J IEEE Transactions on Parallel and Distributed Systems %D 1990 %T IPS-2: the second generation of a parallel program measurement system %A Miller, B. P %A Clark, M. %A Hollingsworth, Jeffrey K %A Kierstead, S. %A Lim,S. -S %A Torzewski, T. %K 4.3BSD UNIX systems %K automatic guidance techniques %K Automatic testing %K Charlotte distributed operating system %K CPA %K DECstation %K design concepts %K distributed programs %K graphical user interface %K Graphical user interfaces %K Instruments %K interactive program analysis %K IPS-2 %K measurement %K message systems %K network operating systems %K Operating systems %K parallel program measurement system %K parallel programming %K parallel programs %K Performance analysis %K performance analysis techniques %K performance evaluation %K performance measurement system %K Power system modeling %K program bottlenecks %K program diagnostics %K Programming profession %K semantics %K Sequent Symmetry multiprocessor %K shared-memory systems %K software tools %K Springs %K Sun %K Sun 4 %K Unix %K VAX %X IPS, a performance measurement system for parallel and distributed programs, is currently running on its second implementation. IPS's model of parallel programs uses knowledge about the semantics of a program's structure to provide two important features. First, IPS provides a large amount of performance data about the execution of a parallel program, and this information is organized so that access to it is easy and intuitive. Secondly, IPS provides performance analysis techniques that help to guide the programmer automatically to the location of program bottlenecks. The first implementation of IPS was a testbed for the basic design concepts, providing experience with a hierarchical program and measurement model, interactive program analysis, and automatic guidance techniques. It was built on the Charlotte distributed operating system. The second implementation, IPS-2, extends the basic system with new instrumentation techniques, an interactive and graphical user interface, and new automatic guidance analysis techniques. This implementation runs on 4.3BSD UNIX systems, on the VAX, DECstation, Sun 4, and Sequent Symmetry multiprocessor %B IEEE Transactions on Parallel and Distributed Systems %V 1 %P 206 - 217 %8 1990/04// %@ 1045-9219 %G eng %N 2 %R 10.1109/71.80132