The block distributed memory model

TitleThe block distributed memory model
Publication TypeJournal Articles
Year of Publication1996
AuthorsJaJa JF, Ryu K W
JournalParallel and Distributed Systems, IEEE Transactions on
Pagination830 - 840
Date Published1996/08//
ISBN Number1045-9219
Keywordsaccesses;single, address, algorithms;performance, algorithms;performance;remote, allocation;sorting;, balancing;matrix, bandwidth;communication, communication;load, complexity;computation, complexity;computational, complexity;cost, complexity;distributed, distributed, evaluation;resource, FFT;block, Fourier, latency;parallel, locality;communication, machines;interconnection, measure;data, memory, model;communication, model;computational, multiplication;memory, multiplication;parallel, problems;distributed, rearrangement, space;sorting;spatial, systems;fast, topology;interprocessor, transforms;matrix

We introduce a computation model for developing and analyzing parallel algorithms on distributed memory machines. The model allows the design of algorithms using a single address space and does not assume any particular interconnection topology. We capture performance by incorporating a cost measure for interprocessor communication induced by remote memory accesses. The cost measure includes parameters reflecting memory latency, communication bandwidth, and spatial locality. Our model allows the initial placement of the input data and pipelined prefetching. We use our model to develop parallel algorithms for various data rearrangement problems, load balancing, sorting, FFT, and matrix multiplication. We show that most of these algorithms achieve optimal or near optimal communication complexity while simultaneously guaranteeing an optimal speed-up in computational complexity. Ongoing experimental work in testing and evaluating these algorithms has thus far shown very promising results