Active semantic caching to optimize multidimensional data analysis in parallel and distributed environments

TitleActive semantic caching to optimize multidimensional data analysis in parallel and distributed environments
Publication TypeJournal Articles
Year of Publication2007
AuthorsAndrade H, Kurc T, Sussman A, Saltz J
JournalParallel Computing
Volume33
Issue7-8
Pagination497 - 520
Date Published2007/08//
ISBN Number0167-8191
KeywordsActive semantic caching, Parallel databases, Query optimization, Scientific data analysis
Abstract

In this paper, we present a multi-query optimization framework based on the concept of active semantic caching. The framework permits the identification and transparent reuse of data and computation in the presence of multiple queries (or query batches) that specify user-defined operators and aggregations originating from scientific data-analysis applications. We show how query scheduling techniques, coupled with intelligent cache replacement policies, can further improve the performance of query processing by leveraging the active semantic caching operators. We also propose a methodology for functionally decomposing complex queries in terms of primitives so that multiple reuse sites are exposed to the query optimizer, to increase the amount of reuse. The optimization framework and the database system implemented with it are designed to be efficient irrespective of the underlying parallel and/or distributed machine configuration. We present experimental results highlighting the performance improvements obtained by our methods using real scientific data-analysis applications on multiple parallel and distributed processing configurations (e.g., single symmetric multiprocessor (SMP) machine, cluster of SMP nodes, and a Grid computing configuration).

URLhttp://www.sciencedirect.com/science/article/pii/S0167819107000506
DOI10.1016/j.parco.2007.03.001