Intermediate Information Structures LECTURE 15

Chain Matrix Multiplication

Jon Rokne

Computer Science

University of Calgary

Canada

CPSC 335

Mostly from: https://www.cse.ust.hk/~dekai/271/notes/L12/L12.pdf‎

Outline of this Lecture

Recalling matrix multiplication.

The chain matrix multiplication problem.

A dynamic programming algorithm for chain matrix multiplication.

2

3

Recalling Matrix Multiplication

4

is a two-

.

5

6

7

8

n  Example: consider the chain A1, A2, A3, A4 of 4 matrices

n  Let us compute the product A1A2A3A4

•  5 different orderings = 5 different parenthesizations

1.  (A1(A2(A3A4)))

2.  (A1((A2A3)A4))

3.  ((A1A2)(A3A4))

4.  ((A1(A2A3))A4)

5.  (((A1A2)A3)A4)

Matrix Chain Multiplication cont..

•  Matrix multiplication is associative , e.g.,

so parenthenization does not change result.

•  To compute the number of scalar multiplications necessary, we must know: •  Algorithm to multiply two matrices •  Matrix dimensions

Algorithm..

Matrix-Multiply(A,B)

1.If columns[A]!=rows[B]

2. then error “incomplete dimensions”

3. else for i 1 to rows[A]

4. do for j 1 to columns[B] 5.  do C[i,j] 0 6.  for k 1 to columns[A] 7.  Do C[i,j]=C[i,j]+A[i,k]*B[k,j]

8. Return C

• The time to compute C is dominated by the number of scalar multiplications. • To illustrate the different costs incurred by different paranthesization of a

matrix product. Example: Consider three matrices A10×100, B100×5, and C5×50 There are 2 ways to parenthesize

((AB)C) = D10×5 · C5×50

AB ⇒ 10·100·5=5,000 scalar multiplications

DC ⇒ 10·5·50 =2,500 scalar multiplications

(A(BC)) = A10×100 · E100×50

BC ⇒ 100·5·50=25,000 scalar multiplications Total: 75,000

AE ⇒ 10·100·50 =50,000 scalar multiplications

Total: 7,500

•  Matrix-chain multiplication problem ü  Given a chain A1, A2, …, An of n matrices, where for i=1, 2, …, n,

matrix Ai has dimension pi-1×pi ü  Parenthesize the product A1A2…An such that the total number of

scalar multiplications is minimized

•  Before solving by Dynamic programming exhaustively check all

paranthesizations. •  P(n) : paranthesization of a sequence of n matrices

Counting the Number of Parenthesizations

• We obtain the recurrence

P nif n

P k p n k if nk

n( ) ( ) ( )==

−∑ ≥

⎧

⎨⎪

⎩⎪ =

−1 1

21

1

1.The Structure of an Optimal Parenthesization Step 1: Characterize the structure of an optimal solution • Ai..j : matrix that results from evaluating the product Ai Ai+1 Ai+2 ... Aj

• An optimal parenthesization of the product A1A2 ... An

– Splits the product between Ak and Ak+1, for some 1≤k<n

(A1A2A3 ... Ak) · (Ak+1Ak+2 ... An)

– i.e., first compute A1..k and Ak+1..n and then multiply these two •  The cost of this optimal parenthesization Cost of computing A1..k + Cost of computing Ak+1..n + Cost of multiplying A1..k · Ak+1..n

17

n  Idea #1: repeatedly select the product that uses the fewest operations.

n  Counter-example: n  A is 101 × 11 n  B is 11 × 9 n  C is 9 × 100 n  D is 100 × 99 n  Greedy idea #1 gives A*((B*C)*D)), which takes

109989+9900+108900=228789 ops n  (A*B)*(C*D) takes 9999+89991+89100=189090 ops

n  The greedy approach is not giving us the optimal value.

Greedy Approach

Is there a better approach? Yes – Dynamic Programming

18

19

20

21

22

We store the subproblems in an array.

23

24

25

26

27

28

29

30

31

32

33

34

35

36

1..4

1..1 2..4 1..2 3..4 1..3 4..4

2..2 3..4 2..3 4..4 3..3 4..4 1..1 2..2

3..3 4..4 2..2 3..3

...

Subproblem Overlap

37

7125}

1137520*10*3504375,712520*5*3510002625,1300020*15*3525000

min{

5414,43,1

5314,32,1

5214,21,1

4,1

=

=++=++

=++=++

=++=++

=

dddNNdddNNdddNN

N

n  A0: 30 X 35; A1: 35 X15; A2: 15X5; A3: 5X10; A4: 10X20; A5: 20 X 25

}{min 11,1,, +++<≤

++= jkijkkijkiji dddNNN

Algorithm Visualization – second example

38

n  A0: 30 X 35; A1: 35 X15; A2: 15X5; A3: 5X10; A4: 10X20; A5: 20 X 25

Algorithm Visualization – second example

(A0*(A1*A2))*((A3*A4)*A5)

39

40

41

42

43

44