Communication-Avoiding Parallel Algorithms for Solving Triangular Matrix Equations
Year: 2015
Type: dissertation
Abstract: In this work an algorithm for solving triangular systems of equations for multiple right hand sides is presented.The algorithm for solving triangular systems for multiple right hand sides, commonly referred to as the TRSM problem, is a very important in dense linear algebra as it is a subroutine for most decompositions of matrices as LU or QR.To improve performance over the standard iterative algorithms for TRSM, a block wise inversion paired with triangular matrix multiplications is used.To perform the inversion, the lower triangular form of the matrix is exploited and a recursive scheme is applied to further decrease communication cost.With that, the latency of the algorithm decreases while the bandwidth and floating point operations count stay asymptotically the same.Concretely, a decrease of latency with a factor of p 2/3 / log p was achieved for a significant range of relative matrix sizes when working with p processors.The proposed method is implemented and its performance is benchmarked against the widely used ScaLAPACK [1] library.The results show promising tendencies for the inversion, with a maximal speedup of 1.7 over ScaLAPACK for 4096 processors.Due to the inferior performance of triangular matrix multiplications with respect to the triangular solve, no overall improvement is made yet.i Motivation: With the decreasing importance of the amount of floating point operations on the total execution time of a program, communication costs will turn out to be a more and more important factor.The goal therefore is to find an algorithm that stays asymptotically optimal in terms of flop-cost while decreasing the communication cost.The approach taken here is to specifically decrease the asymptotic latency cost while keeping the bandwidth constant.We take this approach because we assume that the asymptotic upper bounds for the bandwidth are very strong for the standard recursive approach as described in [2] since they are equal to the bandwidth costs that are required in for matrix multiplication.TRSM is a crucial problem in many applications as it is a subroutine for a lot of algorithms in dense linear algebra, as for example the LU decomposition described in [3], or the QR decomposition.An other very important application of the TRSM is the recursive Cholesky factorization where it is the base-case algorithm as described in [4].It is also the critical routine for solving general dense linear systems of equations.Problem Definition: The linear system of equations L • X = B will be solved for X ∈ R n×k , where L ∈ R n×n denotes a lower-triangular matrix and B ∈ R n×k a dense matrix.In this work we will only account for communication cost (bandwidth and latency) that arises due to communication between different compute units.In the theoretical part of this work, three algorithms are presented to solve the problem recursively where each algorithm takes a different approach: The algorithm referred to as Recursive TRSM splits the initial problem into smaller subproblems, until the problem
Author: Wicky, Tobias
Language: English
Topic: Matrix Theory and Algorithms
Subfield: Computational Theory and Mathematics
Field: Computer Science
Domain: Physical Sciences
Keywords: multiple right-hand sides, communication-avoiding algorithms, recursive algorithm, lower triangular matrices, latency reduction, bandwidth, dense linear algebra, parallel algorithms, LU decomposition, QR decomposition, Cholesky decomposition, matrix multiplication, speedup, fault tolerance
Open Access status: green
OpenAlex ID: W2340409084
