Lecture Notes in

Computer Science
Edited by G. Goos and J. Hartmanis

157
[ III

Ole ~sterby
Zahari Zlatev

Direct Methods
for Sparse Matrices
I I

Springer-Verlag
Berlin Heidelberg New York Tokyo 1983
Editorial Board
D. Barstow W. Brauer P. Brinch Hansen D. Gries D. Luckham
C. Moter A. Pnueli G. SeegmLiller J. Stoer N. Wirth

Authors
Ole t~sterby
Computer Science Department, Aarhus University
DK 8000 Aarhus, Denmark

Zahari Zlatev
Air Pollution Laboratory
Danish Agency of Environmental Protection
Rise National Laboratory
DK 4000 Roskilde, Denmark

CR Subject Classifications (1982): G.1.3

tSBN 3-54042676-7 Springer-Verlag Berlin Heidelberg New York Tokyo

ISBN 0-38742676-7 Springer-Verlag New York Heidelberg Berlin Tokyo

This work is subject to copyright.All rights are reserved,whetherthe whole or part of the material
is concerned, specificallythose of translation,reprinting,re-useof illustrations,broadcasting,
reproduction by photocopying machineor similar means,and storage in data banks. Under
£954 of the German Copyright Law where copies are madefor other than privateuse, a fee is
payableto "VerwertungsgesetlschaftWort", Munich.
© by Springer-VerlagBerlin Heidelberg 1983
Printed in Germany
Printing and binding: Beltz Offsetdruck, Hemsbach/Bergstr.
2145/3140-543210
 Preface

The m a t h e m a t i c a l m o d e l s of m a n y p r a c t i c a l p r o b l e m s lead to systems

of linear a l g e b r a i c e q u a t i o n s where the c o e f f i c i e n t m a t r i x is large
and sparse. Typical e x a m p l e s are the solutions of p a r t i a l d i f f e r e n -
tial e q u a t i o n s by finite d i f f e r e n c e or finite e l e m e n t m e t h o d s but
m a n y other a p p l i c a t i o n s could be mentioned.

When there is a large p r o p o r t i o n of zeros in the c o e f f i c i e n t m a t r i x

then it is fairly obvious that we do not want to store all those
zeros in the computer, but it m i g h t not be quite so o b v i o u s h o w to
get a r o u n d it. We shall first d e s c r i b e storage t e c h n i q u e s w h i c h are
c o n v e n i e n t to use w i t h direct solution methods, and we shall then
show how a very e f f i c i e n t c o m p u t a t i o n a l scheme can be based on Gauss-
Jan e l i m i n a t i o n and iterative refinement.

A serious p r o b l e m in the storage and h a n d l i n g of sparse m a t r i c e s is

the a p p e a r a n c e of fill-ins, i.e. new elements w h i c h are created in
the process of g e n e r a t i n g zeros b e l o w the diagonal. Many of these
new e l e m e n t s tend to be smaller than the o r i g i n a l m a t r i x elements,
and if they are smaller than a certain q u a n t i t y w h i c h we shall call
the drop t o l e r a n c e we simply ignore them. In this way we may pre-
serve the s p a r s i t y quite well but we p r o b a b l y introduce rather large
errors in the LU d e c o m p o s i t i o n to the effect that the solution be-
comes unacceptable. In order to r e t r i e v e the a c c u r a c y we use itera-
tive r e f i n e m e n t and we show t h e o r e t i c a l l y and w i t h p r a c t i c a l experi-
ments that it is ideal for the purpose.

Altogether, the c o m b i n a t i o n of G a u s s i a n elimination, a large drop

tolerance, and i t e r a t i v e r e f i n e m e n t gives a v e r y e f f i c i e n t and compe-
titive c o m p u t a t i o n a l scheme for sparse problems. For dense m a t r i c e s
iterative r e f i n e m e n t will always require more storage and c o m p u t a t i o n
time, and the extra a c c u r a c y it y i e l d s may not be e n o u g h to justify
it. For sparse problems, however, iterative refinement combined with
a large d r o p t o l e r a n c e w i l l in most cases give v e r y accurate results
and r e l i a b l e error e s t i m a t e s with less storage and c o m p u t a t i o n time.
 IV

A short d e s c r i p t i o n of the G a u s s i a n e l i m i n a t i o n process is given in

chapter I. D i f f e r e n t storage a l g o r i t h m s for general sparse m a t r i c e s
are d i s c u s s e d in chapter 2. Chapter 3 is devoted to the use of pivo-
tal strategies as a tool for keeping the b a l a n c e b e t w e e n s p a r s i t y
and accuracy. The p o s s i b i l i t y of using an iterative r e f i n e m e n t proc-
ess in c o n n e c t i o n w i t h the G a u s s i a n e l i m i n a t i o n is the topic of
chapter 4.

In chapter 5 we introduce a general c o m p u t a t i o n a l scheme which in-

cludes m a n y W e l l - k n o w n direct methods for linear e q u a t i o n s and for
o v e r d e t e r m i n e d linear systems as special cases. We also d e m o n s t r a t e
how the above techniques can be g e n e r a l i z e d to linear least squares
problems. Thus, we show that the theory of m o s t of the direct meth-
ods can be studied from a common point of v i e w and that the algo-
rithms d e s c r i b e d in the previous chapters are a p p l i c a b l e not only
in c o n n e c t i o n with Gaussian e l i m i n a t i o n but also for many other
methods. A p a r t i c u l a r a l g o r i t h m (the G e n t l e m a n - Givens o r t h o g o n a -
lization) is d i s c u s s e d in detail in the second part of chapter 5 as
an i l l u s t r a t i o n of the above statements.

The a l g o r i t h m s d e s c r i b e d in chapters 2 - 4 have been i m p l e m e n t e d in

a package for the solution of large and sparse systems of linear al-
gebraic equations. This package, Y12M, is i n c l u d e d in the s t a n d a r d
library at RECKU (the Regional C o m p u t i n g Centre at the U n i v e r s i t y of
Copenhagen). The subroutines of package YI2M with full d o c u m e n t a -
tion and with many t e s t - p r o g r a m s are a v a i l a b l e at the usual cost
(for the m a g n e t i c tape, machine time, shipment, etc.). Requests
should be a d d r e s s e d to J. Wasniewski, RECKU, V e r m u n d s g a d e 5, DK -
2100 Copenhagen. It should be m e n t i o n e d that the subroutines are
w r i t t e n in FORTRAN. Both double and single p r e c i s i o n v e r s i o n s are
available. No special features of the c o m p u t e r at the disposal at
RECKU (UNIVAC 1100/82) have been e x p l o i t e d and no m a c h i n e - d e p e n d e n t
constants are used. Thus the package is p o r t a b l e and will work with-
out any changes on m a n y large computers. This has been v e r i f i e d by
running the subroutines of the package on three d i f f e r e n t computers:
a UNIVAC 1100/82 c o m p u t e r at RECKU, an IBM 3033 computer at the
N o r t h e r n Europe U n i v e r s i t y C o m p u t i n g Centre (NEUCC) and a CDC Cyber
173 computer at the Regional C o m p u t i n g Centre at Aarhus U n i v e r s i t y
(RECAU) .
 V

The package Y12M also includes subroutines for estimation of the con-
dition number of a sparse matrix. The subroutines can be called when
the LU decomposition is calculated and provide a relatively inexpen-
sive but still reliable measure of the sensitivity of the results to
round-off errors.

A full documentation of the subroutines from package Y12M with a

brief description of the basic ideas applied in the implementation is
given in a previous volume of this series (see Z. Zlatev, J. Wasniewski
and K. Schaumburg: "Y12M - Solution of Large and Sparse Systems of
Linear Algebraic Equations", Lecture Notes in Computer Science, Vol.
121, Springer, Berlin-Heidelberg-New York, 1981).

Decimal notation is used for the numbering of sections and chapters.

Thus the third section of chapter 5 is numbered 5.3. The 15th numbered
equation in section 3 of chapter 5 is numbered (3.15) and is referenced
in another chapter by (5.3.15). Tables and figures are numbered in each
chapter. Thus the 7th table or figure in chapter I is numbered 1.7. A
similar numbering system is used for theorems, corollaries, remarks,
etc.

We would like to express our thanks to Angelika Paysen who with great
patience and expert skill typed the manuscript.
 Contents

Preface

I. Introduction

1.1 Gaussian elimination ............................ I

1.2 Sparse matrices ................................. 4
1.3 Test matrices ................................... 6
1.4 An example ...................................... 11
1.5 Contents of c h a p t e r s 2 - 5 ....................... 13

2. Storage Techniques

2.1 Input requirements .............................. 14

2.2 Reordering the structure ........................ 15
2.3 The elimination process ......................... 22
2.4 Storage of f i l l - i n s ............................. 25
2.5 Garbage collections ............................. 30
2.6 On the storage of matrix L ..................... 34
2.7 Classification of p r o b l e m s ...................... 35
2.8 A comparison of o r d e r e d and linked lists ........ 38

3. Pivotal Strategies

3.1 Why interchange rows and columns? ............... 42

3.2 The Markowitz strategy .......................... 44
3.3 The generalized Markowitz strategy (GMS) ........ 45
3.4 The improved generalized Markowitz strategy (IGMS) 48
3.5 Implementation of t h e p i v o t a l strategy .......... 54
3.6 Other strategies ................................ 57
 Vlt

4. Iterative Refinement

4.1 Convergence of iterative refinement ............... 59

4.2 The drop tolerance ................................ 62
4.3 Storage comparisons ............................... 63
4.4 Computing time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5 Choice of d r o p t o l e r a n c e and stability factor ..... 72
4.6 When and how to use iterative refinement .......... 75
4.7 Least squares problems ............................ 78
4.8 Condition number estimation ....................... 82
4.9 Robustness and reliability ........................ 83
4.10 Concluding remarks on IR and T ................. 86

5. Other Direct Methods

5.1 Linear least squares problems ..................... 87

5.2 The general k-stage direct method ................. 89
5.3 Special cases of the general method ............... 93
5.4 Generalized iterative refinement .................. 97
5.5 Orthogonal transformations ........................ 102
5.6 Pivotal strategy .................................. 107
5.7 A 2-stage method based on orthogonal transformations 110
5.8 Numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111

Appendix

Codes for sparse matrix problems .................. 116

References ................................................. 119