Matrix Chain Multiplication

a

b

b

c

=

A = a x b matrixB = b x c matrix

matrix multiplication

Multiplying the Matrix

a

b

b

c

=

Time used = Θ(abc)

Naïve Method

for (i = 1; i <= a;i++) { for (j = 1; i <= c;j++) { sum = 0; for (k = 1;k <= b;k++) { sum += A[i][k] * B[k][j]; } C[i][j] = sum; }}

O(abc)

N x N matrix

N x N matrix

A B C

N x

1 m

atrix

Matrix Chain Multiplication

How to compute ABC ?

Matrix Multiplication

• ABC = (AB)C = A(BC)• (AB)C differs from A(BC)?– Same result, different efficiency

• What is the cost of (AB)C?• What is the cost of A(BC)?

(AB)C

N x N N x N

N x1

N x N

N*N*N

N*N*1

N x1

A(BC)

N x N

N x N N x 1

N*N*1

N*N*1

N x 1

N x 1

The Problem

• Input: a1,a2,a3,….,an

– n-1 matrices of sizes– a1 x a2 B1

– a2 x a3 B2

– a3 x a4 B3

– ….– an-1 x an Bn-1

• Output– The order of multiplication– How to parenthesize the chain

Example

• a1 a2 a3 a4 a5 a6

• 10 x 5 x 1 x 5 x 10 x 2B1 B2 B3 B4 B5

INPUT

Possible Output

((B1B2)(B3B4))B5

(B1((B2B3)B4))B5

(B1B2)((B3B4)B5)

And much more…

Consider the Output

(B1B2)((B3B4)B5)

(B1B2)(B3(B4B5))

What do

((B1B2)(B3B4))B5

(((B1B2)B3)B4))B5

have in common?

What do

have in common?

Min cost of

B1 (B2 B3 B4 … Bn-1 )(B1 B2) (B3 B4 … Bn-1 )(B1 B2 B3) (B4 … Bn-1 )…(B1 B2 B3 B4 …) Bn-1

Solving B1 B2 B3 B4 … Bn-1

Min cost of

B1 (B2 B3 B4 … Bn-1 )(B1 B2) (B3 B4 … Bn-1 )(B1 B2 B3) (B4 … Bn-1 )…(B1 B2 B3 B4 …) Bn-1

Solving B1 B2 B3 B4 … Bn-1

Sub problem

Sub problem

Sub problemSub problem

Sub problem

Sub problem

Matrix Chain Multiplication

B1…BN-1

(B1)(B2…BN-1) (B1B2)(B3…BN-1) (B1…)(BN-1)

(B…)(B2…BN-2)… … … …

…

(B2…BN-2) (BN-1)

Deriving the Recurrent

• mcm(l,r)– The least cost to multiply Bl … Br

• The solution is mcm(1,n-1)

The Recurrence

• Initial Case– mcm(x,x) = 0– mcm(x,x+1) = a[x] * a[x+1] * a[x+2]

The Recurrence

min cost of Bl (Bl+1 Bl+2 Bl+3 … Br ) + al•al+1•ar+1

min cost of (Bl Bl+1) (Bl+2 Bl+3 … Br ) + al•al+2•ar+1

min cost of (Bl Bl+1 Bl+2) (Bl+3 … Br ) + al•al+3•ar+1

…

min cost of (Bl Bl+1 Bl+2 Bl+3 …) Br + al•ar•ar+1

mcm(l,r) = min of

The Recurrence

Bl (Bl+1 Bl+2 Bl+3 … Br ) + al•al+1•ar+1

(Bl Bl+1) (Bl+2 Bl+3 … Br ) + al•al+2•ar+1

(Bl Bl+1 Bl+2) (Bl+3 … Br ) + al•al+3•ar+1

…

(Bl Bl+1 Bl+2 Bl+3 …) Br + al•ar•ar+1

mcm(l,r) = min of

mcm(l+1,r)

mcm(l+2,r)

mcm(l+3,r)mcm(l,l+2)

mcm(l,l+1)

mcm(l,r-1)

0

0

+

+

+

+ +

+

+

+

Matrix Chain Multiplication

int mcm(int l,int r) { if (l < r) { minCost = MAX_INT; for (int i = l;i < r;i++) { my_cost = mcm(l,i) + mcm(i+1,r) + (a[l] * a[i+1] * a[r+1]);

minCost = min(my_cost,minCost); } return minCost; } else { return 0; }}

Using bottom-up DP

• Design the table

• M[i,j] = the best solution (min cost) for multiplying Bi…Bj

• The solution is at M[i,n-1]

What is M[i,j]?

• Trivial case– What is m[x,x] ?– No multiplication, m[x,x] = 0

What is M[i,j]?

• Simple case– What is m[x,x+1] ?– BxBx+1

– Only one solution = ax * ax+1 * ax+2

What is M[i,j]?

• General case– What is m[x,x+k] ?– BxBx+1Bx+2…Bx+k

Bx (Bx+1 Bx+2 Bx+3 … Bx-k ) + ax•ax+1•ax+k+1

(Bx Bx+1) (Bx+2 Bx+3 … Bx+k ) + ax•ax+2•ax-k+1

(Bx Bx+1 Bx+2) (Bx+3 … Bx+k ) + ax•ax+3•ax+k+1

…

(Bx Bx+1 Bx+2 Bx+3 …) Bx+k + ax•ax+k•ax+k+1

min of

Filling the Table

1 2 3 4 5 6

1

2

3

4

5

6

M[1,1]

M[1,6] (our

solution)

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Trivial case

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Arbitrary case

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Arbitrary case

Plus a1•a2•a6

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Arbitrary case

Plus a1•a3•a6

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Arbitrary case

Plus a1•a4•a6

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Arbitrary case

Plus a1•a5•a6

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

Filling the Table

1 2 3 4 5 6

1 0

2 0

3 0

4 0

5 0

6 0

12

34

5

Example

• a1 a2 a3 a4 a5 a6

• 10 x 5 x 1 x 5 x 10 x 2B1 B2 B3 B4 B5

Example

1 2 3 4 5

1 0

2 0

3 0

4 0

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0

3 0

4 0

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0 25

3 0

4 0

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0 25

3 0 50

4 0

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 1 = 0 + 25 + 10 x 5 x 5 = 275

Example

1 2 3 4 5

1 0 50

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 2 = 50 + 25 + 10 x 1 x 5 = 100 minimal

Example

1 2 3 4 5

1 0 50 100(2)

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 2 = 50 + 25 + 10 x 1 x 5 = 100 minimal

(2) means that the minimal

solution is by dividing at B2

Example

1 2 3 4 5

1 0 50 100(2)

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 1 = 0+ 50 + 5x 1 x 10 = 100

Example

1 2 3 4 5

1 0 50 100(2)

2 0 25

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 1 = 25+ 0 + 5x 5 x 10 = 275

Example

1 2 3 4 5

1 0 50 100(2)

2 0 25 100(2)

3 0 50

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Option 1 is better

Example

1 2 3 4 5

1 0 50 100(2)

2 0 25 100(2)

3 0 50 70(4)

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50 100(2)

200(2)

2 0 25 100(2)

3 0 50 70(4)

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50 100(2)

200(2)

2 0 25 100(2)

80(2)

3 0 50 70(4)

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2

Example

1 2 3 4 5

1 0 50 100(2)

200(2)

140(2)

2 0 25 100(2)

80(2)

3 0 50 70(4)

4 0 100

5 0

a1 a2 a3 a4 a5 a6 10 x 5 x 1 x 5 x 10 x 2