Necessity is the mother of invention: a simple grid computing system using commodity tools

TitleNecessity is the mother of invention: a simple grid computing system using commodity tools
Publication TypeJournal Articles
Year of Publication2003
AuthorsMyers DS, Cummings MP
JournalJ Parallel Distr Com
Pagination578 - 589
Date Published2003/05//
KeywordsApache;, computing;, distributed, Grid, HTTP;, Java;, Linux;, Perl;, SQL;, Unix;, XML-RPC

Access to sufficient resources is a barrier to scientific progress for many researchers facing large computational problems. Gaining access to large-scale resources (i.e., university-wide or federally supported computer centers) can be difficult, given their limited availability, particular architectures, and request/review/approval cycles. Simultaneously, researchers often find themselves with access to workstations and older clusters overlooked by their owners in favor of newer hardware. Software to tie these resources into a coherent Grid, however, has been problematic. Here, we describe our experiences building a Grid computing system to conduct a large-scale simulation study using "borrowed" computing resources distributed over a wide area. Using standard software components, we have produced a Grid computing system capable of coupling several hundred processors spanning multiple continents and administrative domains. We believe that this system fills an important niche between a closely coupled local system and a heavyweight, highly customized wide area system. (C) 2003 Elsevier Science (USA). All rights reserved.