ECE 530 – Analysis Techniques for Large-Scale Electrical Systems

Prof. Hao Zhu

Dept. of Electrical and Computer Engineering

University of Illinois at Urbana-Champaign

haozhu@illinois.edu

9/30/2014

1

Lecture 11: Optimal Ordering of Sparse Systems

Announcements

• HW 4 is due Oct 10 (Friday)

• Midterm exam is October 23 in class; closed book and notes, but one 8.5 by 11 inch note sheet and simple calculators allowed

• Reference: W.F. Tinney and J.W. Walker, “Direct solutions of sparse network equations by optimally ordered triangular factorization,” Proc. IEEE, vol. 55, pp. 1801-1809, November 1967.

2

• We make use of graph theoretic notions to develop a practical reordering scheme for sparse systems

• We associate a graph G with the zero-nonzero structure of the n by n matrix A

• We construct the graph G associated with the matrix A as follows: i. G has n nodes corresponding to the dimension n of the

square matrix: node i represents both the column i and the row i of A;

ii. a branch (k, j) connecting nodes k and j exists if and only if the element Ajk (and, by structural symmetry, Akj) is nonzero; the self loop corresponding to Akk is not represented

Graph Associated with A

3

Example: Linear Resistive Network

• We consider the simple linear resistive network with the topology shown below

1s 2s

4s 3s4

• The corresponding loop impedance matrix has the following zero - nonzero pattern:

1 2 3 4

1 X X 0 X

2 X X X 0

3 0 X X X

4 X 0 X X

rc

5

Example: Linear Resistive Network

Example: Linear Resistive Network

• The associated 4-node graph of the matrix is

• We next illustrate the graphical interpretation of the elimination process with another example

6

1 2

3 4

Example: 5 by 5 System

• Suppose that A has the zero-nonzero pattern

7

1 2 3 4 5

1 X X X X

2 X X X

3 X X X

4 X X X X

5 X X X

rc

Example: 5 by 5 System

• Then, the associated graph G is

8

1 2

34

5

• We eliminate the Bus (Node) 1 variable with the resulting zero-nonzero pattern as shown the array

bordered by the dashed lines:

the new associated graph G1

1 2 3 4 5

1 X X X X

2 X X X F F

3 X X X

4 X F X X X

5 X F X X

rc

Example: Eliminating Bus 1

9

Graph G1

We obtain the graph G1 from G by removing Bus 1 (corresponding to the eliminating of “ x1 ”) with the new added branches (2, 4) and (2, 5) corresponding to the fills

5 2

4 3

new branch

Example: New Graph G1

10

The elimination of Bus 2 results in the submatrix

3 4 5

3 X X F

4 X X X

5 F X X

rc

Example: Eliminating Bus 2

11

with the corresponding graph G2 5

4 3

The elimination of Bus 3 yields

4 5

4 X X

5 X X

rc

Example: Eliminating Bus 3

12

with the corresponding graph G3

5

4

Finally, upon Bus 4 we have

and the corresponding G4 is simply the one-node graph

5

5 X

5

rc

Example: Eliminating Bus 4

13

• The graph-theoretic interpretation of the elimination of the node (bus) j (or equivalently row j) is as follows

• The deletion of the node j involves all its incident branches (k, j) and (j, k) connected to j, for all k j

• In the pre-elimination graph of the eliminated node j, the elimination of the branches (j, k) and (l, j) results in the addition of the new branch (k, l), if one does not already exist

Graph-Theoretic Interpretation

14

• We next examine how we may reorder the rows and columns of A to preserve its sparsity, i.e., to minimize the number of fills

• Eventually we’ll introduce an algorithm to try to minimize the fills

• This is motivated by revisiting the 5-node graph G

1 2

34

5

Reording the Rows/Columns

15

• To minimize the number of fills, i.e., the number of new branches in G, we eliminate first the node which upon deletion introduces the least number of new branches

• This is node 5 and upon deletion no new branches are added and the resulting graph G1 is

1

4

2

3

16

Motivating Example

• The structure of G1 is such that any one of the remaining nodes may be chosen as the next node to be eliminated since each of the 4 remaining nodes introduces a new branch after its elimination

• We arbitrarily pick node 1 and we obtain the graph G2• We continue with the next three choices arbitrarily,

none resulting in new fills2

34

new branch

17

Motivating Example

• We may relabel the original graph in such a way that the label of the node refers to the order in which it was eliminated

• Thus we renumber the nodes as shown below

12 3

45

18

Motivating Example

• Clearly, relabeling the nodes corresponds to reordering the rows and columns of A

• For the reordered system, the zero-nonzero pattern of A becomes

1 2 3 4 5

1 X X X

2 X X X X

3 X X X

4 X X X

5 X X X X

rc

19

Motivating Example

and the post-elimination matrix has the zero-nonzero structure

• Compared to the original ordering scheme, the new ordering scheme has saved us 4 fill-ins

1 2 3 4 5

1 X X X

2 X X X X

3 X X X F

4 X X X

5 X X F X X

rc

20

Motivating Example

• The associated graph of the structurally symmetric matrix A is useful in gaining insights into the factorization process

• We make the following observations• If A is originally structurally symmetric, then it

remains so in all the steps of the factorization;• a good ordering scheme is independent of the

values of the elements of A and depends only on its the zero-nonzero pattern

21

General Findings

22

Matrix Recording

• The problem of ordering the columns and rows of so that the elimination results in the least number of zero-nonzero terms in the table of factors – the so called globally optimal ordering problem – remains largely unsolved

• A scheme which orders the rows and columns of A such that at each step of the process the row and the column chosen next is the one that minimizes the number of fill-ins in that step is a locally optimal ordering scheme and is usually referred to, albeit incorrectly, as the optimal ordering

Locally Optimal Recording Scheme

• Suppose we are at the stage where we have picked the order of the first m – 1 rows that are eliminated

• At this stage, we have the zero-nonzero pattern of all the derived elements , for all the rows below the first (m – 1) rows, i.e., k > m-1 and we focus on the rest submatrix

• We want to determine which one of the remaining

n – (m – 1) = n-m+1

rows to pick as the m-th row in the elimination process• Suppose we pick row l as the m-th row of elimination

23

, 1ik ja i m

At the m-th Elimination Stage

24

1

1 1 1 1 1 11 1 1

1 1 1 1 1 11 1 1

1 1 1 1 1 11 1 1 1 1 1 1 1

11 1

1

1

1

m m m m m mm m n

m m m m m mm m m m m m m m n

m m m m m mm m n

m

a a a a a a

a a a a a a

a a a a a a

a a

1 1 1 1 11 1 1 1 1 1 1

1 1 1 1 1 11 1 1

m m m m mm m n

m m m m m mn n m n m n n n n

a a a a

a a a a a a

m

1

2

1

Eliminating Row l

• So relabel row l and column l as the m-th row and the m-th column, respectively

• The modification introduced in the rows below row m are

25

, , , , , , ,

1 1

1 2 1 2

m m m mk j k j k ja a a a

k m m n j m m n k j

Fill-in

• A fill-in after this elimination is defined as

• From the motivating example, a fill-in is introduced in the position (k, j) by eliminating row l if and only if

, and both and • Further define the indicator variables

26

1m mk j k ja 0 a 0

1mk ja 0 1m

k a , , 1m m

j ja a 0 i.e.

1

1

1 mk j

k j mk j

a 0

a 00

Number of Fill-ins

• Hence, row l as the m-th row to be eliminated introduces a fill-in in the position (k, j) if and only if

or equivalently,

• The total number of fill-ins introduced in the remaining (n – m) rows (except for row l)

27

1 1k j k j

, 1 1k j k jand 0

,

, , , ... ,

1

1k j k jk j

k jk j m m n

F

An Upper Bound

• Ignoring the last multiplier

28

, ..., 1,...,

,, , ... ,

1

1

1k j k j k jk j k j

k m n j m n

k jk j m n

F

• Denote as the number of nonzero entries in row l of the U matrix at the m-th elimination stage

• So the m-th row to eliminate would be the row l with the smallest amongst the remaining (n-m+1) rows

U

21F U

U

“Optimal” Ordering

• Graph-theoretic interpretations:– in the reduced graph, obtained after eliminating the first (m –

1) nodes, choose as the next node to be eliminated the one with the least number of incident branches

– relabel that node as node m

• We refer to it as Tinney Reordering Scheme 2 • In many situations, we cannot do much better than this

sub-optimal ordering• Often, the terminology used is imprecise: when

someone refers to optimal ordering, the reference is, indeed, to this sub-optimal ordering

29