The Block Distributed Memory Model

Joseph F. JJ

Kwan Woo Ryu

Dept. of Electrical EngineeringDept. of Computer Engineering
Inst. for Advanced Comp. StudiesKyungpook Natl Univ., KOREA
Inst. for Systems ResearchInst. for Advanced Comp. Studies
University of Maryland
College Park, MD 20742

Abstract

We introduce a computation model for developing and analyzing parallel algorithms on distributed memory machines. The model allows the design of algorithms using a single address space and does not assume any particular interconnection topology. We capture performance by incorporating a cost measure for interprocessor communication induced by remote memory accesses. The cost measure includes parameters reflecting memory latency, communication bandwidth, and spatial locality. Our model allows the initial placement of the input data and pipelined prefetching.

We use our model to develop parallel algorithms for various data rearrangement problems, load balancing, sorting, FFT, and matrix multiplication. We show that most of these algorithms achieve optimal or near optimal communication complexity while simultaneously guaranteeing an optimal speed-up in computational complexity.

• Introduction
• The Block Distributed Memory (BDM) Model
• Basic Algorithms for Data Movements
• Load Balancing
• Sorting, FFT, and Matrix Multiplication
  • Sorting
  • Fast Fourier Transform
  • Matrix Multiplication
• References
• About this document ...