動態規劃動態規劃 (( 一一 ))

HKOI Training 2007HKOI Training 2007Kelly ChoiKelly Choi19 May 200719 May 2007

Acknowledgement:Acknowledgement:References and slides extracted fromReferences and slides extracted from1.1. [Advanced] Dynamic Programming, 24-04-2004, by cx[Advanced] Dynamic Programming, 24-04-2004, by cx2.2. [Intermediate] Dynamic Programming, 21-08-2004, by Ng Tung[Intermediate] Dynamic Programming, 21-08-2004, by Ng Tung

PrerequisitePrerequisite

►Concepts in RecurrenceConcepts in Recurrence►Basic RecursionBasic Recursion►FunctionsFunctions►Divide-and-conquerDivide-and-conquer

Grid Path CountingGrid Path Counting

► In a N*M grid, we want to move from the top left In a N*M grid, we want to move from the top left cell to the bottom right cellcell to the bottom right cell

► You may only move You may only move downdown or or rightright► Some cells may be impassableSome cells may be impassable► Example of one path:Example of one path:

Naïve Algorithm: DFSNaïve Algorithm: DFS► Function DFS(x,y: integer) : integer;Function DFS(x,y: integer) : integer;

BeginBeginIf (x <= N) and (y <= M) Then BeginIf (x <= N) and (y <= M) Then Begin

If (x = N) and (y = M) ThenIf (x = N) and (y = M) ThenDFS := 1 DFS := 1 {base case}{base case}

Else If Grid[x,y] <> IMPASSABLE ThenElse If Grid[x,y] <> IMPASSABLE ThenDFS := DFS(x+1,y) + DFS(x,y+1)DFS := DFS(x+1,y) + DFS(x,y+1)

ElseElseDFS := 0; DFS := 0; {impassable cell}{impassable cell}

EndEndEndEnd

Time ComplexityTime Complexity

►Time complexity of this algorithm:Time complexity of this algorithm: Each time the procedure branches into Each time the procedure branches into

two pathstwo paths Time complexity is exponentialTime complexity is exponential

►Alternate method to estimate runtimeAlternate method to estimate runtime The base case is reached the number of The base case is reached the number of

times of paths, so time complexity is at times of paths, so time complexity is at least = number of pathsleast = number of paths

►Have we done anything redundant?Have we done anything redundant?

Slow...Slow...

DFS(1,2)

DFS(1,3) DFS(2,2)

DFS(1,4) DFS(2,3) DFS(3,2)DFS(2,3)

DFS(1,1)

DFS(2,1)

DFS(2,2) DFS(3,1)

DFS(3,2)DFS(2,3)

Overlapping Subproblems Overlapping Subproblems

►Note that DFS(Note that DFS(22,2) ,2) iis called twis called twicce, e, DFS(2,3) three times, etc.DFS(2,3) three times, etc.

►But the But the wwork perork perfformeormedd by these by these DFS(2,2) DFS(2,2) ccalls are alls are unaffected by what unaffected by what called themcalled them, , aand thus are nd thus are reduredunndantdant

►We We ccan an mememorize the values morize the values tthheese se calls calls rreturn, and avoid eturn, and avoid tthe reduhe redunndant dant workwork

Memo(r)izationMemo(r)ization

► Compute and store the value of DFS(x,y) in Compute and store the value of DFS(x,y) in the first time we called it. Afterwards, the first time we called it. Afterwards, retrieve the value of DFS(x,y) from the table retrieve the value of DFS(x,y) from the table directly, without calling DFS(x+1,y) and directly, without calling DFS(x+1,y) and DFS(x,y+1) againDFS(x,y+1) again

► This is called recursion with This is called recursion with memo(r)ization.memo(r)ization.► Time complexity is reduced to O(NM). Time complexity is reduced to O(NM).

(why?)(why?)

Example : Grid Path CountingExample : Grid Path Counting► Function DFS(x,y: integer) : integer;Function DFS(x,y: integer) : integer;

BeginBeginIf (x <= N) and (y <= M) Then BeginIf (x <= N) and (y <= M) Then Begin

If Memory[x,y] = -1 Then BeginIf Memory[x,y] = -1 Then BeginIf (x = N) and (y = M) ThenIf (x = N) and (y = M) Then

Memory[x,y]Memory[x,y] := 1 := 1Else If Grid[x,y] <> IMPASSABLE ThenElse If Grid[x,y] <> IMPASSABLE Then

Memory[x,y]Memory[x,y] := DFS(x+1,y) + := DFS(x+1,y) + DFS(x,y+1)DFS(x,y+1)

ElseElseMemory[x,y]Memory[x,y] := 0; := 0;

EndEndDFS := Memory[x,y];DFS := Memory[x,y];

EndEndEndEnd

Bottom-up approachBottom-up approach

► Iterative, Without the use of function callsIterative, Without the use of function calls► Consider the arrays Grid[x,y] and Memory[x,y]:Consider the arrays Grid[x,y] and Memory[x,y]:

66 33 33 11 00

33 00 22 11 11

33 22 11 00 11

11 11 11 11 11

► Treat DFS(x,y) not as a function, but as an arrayTreat DFS(x,y) not as a function, but as an array► Evaluate the values for DFS[x,y] row-by-row, Evaluate the values for DFS[x,y] row-by-row,

column-by-columncolumn-by-column

Example : Grid Path CountingExample : Grid Path Counting► DFS[N,M] := 1;DFS[N,M] := 1;

For x := 1 to N DoFor x := 1 to N Do If Grid[x,M] = IMPASSABLE Then DFS[x,M] := 0 Else If Grid[x,M] = IMPASSABLE Then DFS[x,M] := 0 Else DFS[x,M] := 1;DFS[x,M] := 1;For y := 1 to M DoFor y := 1 to M Do If Grid[N,y] = IMPASSABLE Then DFS[N,y] := 0 Else DFS[N,y] If Grid[N,y] = IMPASSABLE Then DFS[N,y] := 0 Else DFS[N,y] := 1;:= 1;

For x := N-1 downto 1 DoFor x := N-1 downto 1 DoFor y := M-1 downto 1 DoFor y := M-1 downto 1 Do

If Grid[x,y] = IMPASSABLE ThenIf Grid[x,y] = IMPASSABLE ThenDFS[x,y] := 0;DFS[x,y] := 0;

ElseElseDFS[x,y] := DFS[x+1,y] + DFS[x,y+1];DFS[x,y] := DFS[x+1,y] + DFS[x,y+1];

Computation OrderComputation Order

► Top-down approachTop-down approach Recursive function calls with memo(r)izationRecursive function calls with memo(r)ization Easy to implementEasy to implement Can avoid calculating values for impossible Can avoid calculating values for impossible

statesstates Further optimization possibleFurther optimization possible

► Bottom-up approachBottom-up approach Easy to code once the order is identifiedEasy to code once the order is identified Avoid function callsAvoid function calls

►We usually prefer the bottom-up approachWe usually prefer the bottom-up approach► But we need to see the dependence of But we need to see the dependence of

states on other states to decide the orderstates on other states to decide the order

Components of DPComponents of DP

► It is very important to be able to It is very important to be able to describe a DP algorithmdescribe a DP algorithm

►Two essential components of DPTwo essential components of DP State(State( 狀態狀態 ) and the state value: ) and the state value:

►State State –– a description of the subproblem a description of the subproblem►State value State value –– the value we need according to the value we need according to

the subproblem, usually optimal in some the subproblem, usually optimal in some sense, but can be defined flexibly.sense, but can be defined flexibly.

A rule describing the relationship between A rule describing the relationship between states (states ( 狀態轉移方程狀態轉移方程 ), with base cases), with base cases

Grid Path CountingGrid Path Counting

► In the above problem, the state is the In the above problem, the state is the position – (x,y)position – (x,y)

► The state value is defined as The state value is defined as N(x,y) = Number of paths from that position N(x,y) = Number of paths from that position

to the destinationto the destination

► Formula for State Transfer Formula for State Transfer

N(x,y) =N(x,y) =

1, x=N and y = M1, x=N and y = M

0, Grid[x,y] is impassable0, Grid[x,y] is impassable

N(x+1,y) + N(x,y+1), otherwiseN(x+1,y) + N(x,y+1), otherwise

Defining StatesDefining States

►Defining a good state is the key to solving a Defining a good state is the key to solving a DP problemDP problem Formula for State Transfer comes up easily if we Formula for State Transfer comes up easily if we

have a good definition for the state and the state have a good definition for the state and the state value.value.

The state definition affects the dimension of the The state definition affects the dimension of the problem and thus the time complexityproblem and thus the time complexity

► The state should include all information The state should include all information relevant to the computation of the value. relevant to the computation of the value. Recall what a “Recall what a “functionfunction” means in the ” means in the mathematical sense.mathematical sense.

Defining StatesDefining States

► In optimization problems, we usually In optimization problems, we usually define the state value to be the define the state value to be the optimal value subject to a set of optimal value subject to a set of constraints (i.e. the states). constraints (i.e. the states). Sometimes these constraints simply Sometimes these constraints simply come from varying the constraints come from varying the constraints from the problem, but sometimes from the problem, but sometimes there are better definitions. there are better definitions.

Features of DP ProblemsFeatures of DP Problems

►Overlapping Overlapping sub-problems (sub-problems ( 重疊子問題重疊子問題 ))►Optimal Substructure (Optimal Substructure ( 最優子結構最優子結構 ))►Memoryless Property (Memoryless Property ( 無後效性無後效性 ))

The future depends only on the present, but The future depends only on the present, but not the past. not the past.

i.e. Decision leading to the subproblem does i.e. Decision leading to the subproblem does not affect how we solve the subproblem, not affect how we solve the subproblem, once the subproblem is specified.once the subproblem is specified.

Memo(r)ization VS DPMemo(r)ization VS DP

►Memo(r)ization – the method for Memo(r)ization – the method for speeding up computations by storing speeding up computations by storing previously computed results in previously computed results in memorymemory

►Dynamic Programming – the process Dynamic Programming – the process of setting up and evaluating a of setting up and evaluating a recurrence relation efficiently by recurrence relation efficiently by employing memo(r)izationemploying memo(r)ization

Triangle (IOI ’94)Triangle (IOI ’94)

►Given a triangle with Given a triangle with NN levels like the one on the levels like the one on the left, find a path with left, find a path with maximum sum from the top maximum sum from the top to the bottomto the bottom

►Only the sum is requiredOnly the sum is required

7

6 9

8 1 4

2 4 5 3

Triangle Triangle : A: Analysisnalysis

►Exhaustion? Exhaustion? How many paths are there in total?How many paths are there in total?

►Greedy?Greedy? It doesn’t work. Why?It doesn’t work. Why?

►Graph problem?Graph problem? Possible, but not simple enoughPossible, but not simple enough Fail to make use of the special shape of Fail to make use of the special shape of

the graphthe graph

Triangle: Defining the StateTriangle: Defining the State

►We need to find the maximum sum on We need to find the maximum sum on a path from (1,1) to the bottoma path from (1,1) to the bottom

►We can attempt to define the state by We can attempt to define the state by the cell (the cell (ii,,jj) and state value F[) and state value F[ii][][jj] to be ] to be the maximum sum from (the maximum sum from (ii,,jj) to be ) to be bottombottom

TriangleTriangle: F: Formulationormulation

►Let A[Let A[ii][][jj] denote the number in the ] denote the number in the ii--th row and th row and jj-th column, (-th column, (ii, , jj))

►Let F[Let F[ii][][jj] denote the maximum sum of ] denote the maximum sum of the numbers on a path from the numbers on a path from tthe he bottom to (bottom to (ii, , jj))

►Answer = F[1][1]Answer = F[1][1]►The problem exhibits optimal The problem exhibits optimal

substructure heresubstructure here

TriangleTriangle: F: Formulationormulation

►Base cases: F[Base cases: F[NN][][ii] = A[] = A[NN][][ii] for 1] for 1≤ ≤ ii ≤≤NN

►State Transfer: (State Transfer: (ii < < NN, 1 , 1 ≤≤ jj ≤≤ ii) ) F[F[ii][][jj] = max{F[] = max{F[ii+1][+1][jj],F[],F[ii+1][+1][jj+1]}+A[+1]}+A[ii][][jj] ] 7

6 9

8 1 4

TriangleTriangle: Computation Order: Computation Order

►F[F[ii][*] depends on F[][*] depends on F[ii+1][*]+1][*]►Compute F row by row, from bottom to Compute F row by row, from bottom to

toptop

TriangleTriangle: Algorithm: Algorithm

►Algorithm:Algorithm: for for ii 1 to 1 to NN do do

F[F[NN][][ii] ] A[ A[NN][][ii] ;] ;

for for ii NN-1 downto 1 do-1 downto 1 do

for for jj 1 to 1 to ii do do

F[F[ii][][jj] ] max{F[ max{F[i+1i+1][][jj],F[],F[i+1i+1]][[jj+1]}+1]}

+ A[+ A[ii][][jj]]

answer answer F[1][1] F[1][1]

TriangleTriangle: Complexity: Complexity

►Number of array entries: Number of array entries: NN((NN+1)/2+1)/2►Time for computing one entry: O(1)Time for computing one entry: O(1)►Thus the total time complexity is O(Thus the total time complexity is O(NN22))►Memory complexity: O(Memory complexity: O(NN22))

Triangle: BacktrackingTriangle: Backtracking

►What if we are asked to output the path?What if we are asked to output the path?►We can store the optimal decision, i.e. We can store the optimal decision, i.e.

left or right to go for each cellleft or right to go for each cell► We then travel from (1,1) down to the We then travel from (1,1) down to the

bottom to obtain the pathbottom to obtain the path►This is called Backtracking, “back” in the This is called Backtracking, “back” in the

sense that we go in the reversed order sense that we go in the reversed order of which we obtained the answer. of which we obtained the answer.

TriangleTriangle

► If you refer to past training notes, If you refer to past training notes, there is another definition of the state there is another definition of the state value F[value F[ii][][jj] ] –– the maximum sum of a the maximum sum of a path from (1,1) to (path from (1,1) to (ii,,jj))

►Both algorithm gives the same time Both algorithm gives the same time complexity. But the second definition complexity. But the second definition (from the top to bottom) probably (from the top to bottom) probably gives you some insights for gives you some insights for Triangle IITriangle II on the HKOI Online Judge. Think about on the HKOI Online Judge. Think about it. it.

FishingFishing

►There are There are NN fish ponds and you are fish ponds and you are going to spend going to spend MM minutes on fishing minutes on fishing

►Given the Given the time-rewardtime-reward relationship of relationship of each pond, determine the time you each pond, determine the time you should spend at each pond in order to should spend at each pond in order to get the biggest rewardget the biggest reward

time/pondtime/pond 11 22 33

1 minute1 minute 0 fish0 fish 2 fish2 fish 1 fish1 fish

2 minutes2 minutes 3 fish3 fish 2 fish2 fish 2 fish2 fish

3 minutes3 minutes 3 fish3 fish 4 fish4 fish 4 fish4 fish

Fishing (example)Fishing (example)

►For example, if N=3, M=3 and the For example, if N=3, M=3 and the relationships are given in the previous relationships are given in the previous slide, then the optimal schedule isslide, then the optimal schedule is Pond 1: 2 minutesPond 1: 2 minutes Pond 2: 1 minutePond 2: 1 minute Pond 3: 0 minutePond 3: 0 minute Reward: 5 fishReward: 5 fish

Fishing (analysis)Fishing (analysis)

►You can think of yourself visiting ponds 1, 2, You can think of yourself visiting ponds 1, 2, 3, …, 3, …, NN in order in order Why?Why?

►Suppose in an optimal schedule you spend Suppose in an optimal schedule you spend KK minutes on fishing at pond 1minutes on fishing at pond 1

►So you have So you have MM--KK minutes to spend at the minutes to spend at the remaining remaining NN-1 ponds-1 ponds The problem is reducedThe problem is reduced

►But how can I know what is But how can I know what is KK?? You don’t know, so try all possible values!You don’t know, so try all possible values!

Fishing (formulation)Fishing (formulation)

►Let F[Let F[ii][][jj] be the maximum reward you ] be the maximum reward you can get by spending can get by spending jj minutes at the minutes at the first first ii ponds ponds

►Base cases: 0 ≤ Base cases: 0 ≤ ii, , jj ≤ ≤ NN

F[F[ii][0] = 0][0] = 0

F[0][F[0][jj] = 0] = 0►Progress: 1 ≤ Progress: 1 ≤ ii ≤ ≤ NN, 1 ≤ , 1 ≤ jj ≤ ≤ MM

F[F[ii][][jj] = max{F[] = max{F[ii-1][-1][kk]+R[]+R[ii][][jj--kk]}]}0 ≤ k ≤ j

StampsStamps

►Given an unlimited supply of $2, $3 Given an unlimited supply of $2, $3 and $5 stamps, find the number of and $5 stamps, find the number of different ways to form a denomination different ways to form a denomination of $of $NN

►Example:Example: NN = 10 = 10 {2, 2, 2, 2, 2}, {2, 2, 3, 3}, {2, 3, 5},{2, 2, 2, 2, 2}, {2, 2, 3, 3}, {2, 3, 5},

{5, 5}{5, 5}

Polygon TriangulationPolygon Triangulation

►Given an Given an NN--sided convex sided convex polygon polygon AA, find , find a triangulation a triangulation scheme with scheme with minimum total minimum total cut lengthcut length

1

2

34

5

6

Polygon (analysis)Polygon (analysis)

►Every edge of A belongs to exactly one Every edge of A belongs to exactly one triangle resulting from the triangulationtriangle resulting from the triangulation

1

2

4

5

6

4

1

34

2

►We get two We get two (or one) (or one) smaller smaller polygons polygons after after deleting a deleting a triangletriangle

Polygon (analysis)Polygon (analysis)

►The order of cutting does not matterThe order of cutting does not matter►Optimal substructureOptimal substructure

If the cutting scheme for A is optimal, If the cutting scheme for A is optimal, then the cutting schemes for B and C then the cutting schemes for B and C must also be optimalmust also be optimal

A

B

C

Best Flying Route (Best Flying Route ( 最佳航線問最佳航線問題題 11))

► Cities are numbered from 1 to Cities are numbered from 1 to NN, from West to East, from West to East► There are some one-way flights connecting pairs of There are some one-way flights connecting pairs of

citiescities► You want to fly from city 1 to city You want to fly from city 1 to city NN, always flying to , always flying to

the East, then back to city 1, always flying to the the East, then back to city 1, always flying to the WestWest

► Each city, except city 1, can be visited at most onceEach city, except city 1, can be visited at most once► e.g. When N = 10, e.g. When N = 10,

You can fly in the sequence {1, 2, 5, 8, 10, 7, 3, 1} (if there You can fly in the sequence {1, 2, 5, 8, 10, 7, 3, 1} (if there are flights connecting each adjacent pair of them), but not in are flights connecting each adjacent pair of them), but not in the sequence {1, 4, 2, 5, 10, 1}, nor {1, 3, 5, 10, 5, 1}, even the sequence {1, 4, 2, 5, 10, 1}, nor {1, 3, 5, 10, 5, 1}, even if there are flights connectin each adjacent pair of them.if there are flights connectin each adjacent pair of them.

► Assume at least one such route existsAssume at least one such route exists► Maximize the number of cities you can visit within the Maximize the number of cities you can visit within the

above constraintsabove constraints

Best Flying RouteBest Flying Route

►Does the problem demonstrate optimal Does the problem demonstrate optimal substructure? Are the states memoryless?substructure? Are the states memoryless? Unfortunately, after flying from 1 to Unfortunately, after flying from 1 to NN, how we , how we

can fly back to 1 depends on what cities we can fly back to 1 depends on what cities we visited in flying to visited in flying to NN

How to formulate the problem? How are How to formulate the problem? How are subproblems related?subproblems related?

►Time Complexity and Memory ComplexityTime Complexity and Memory Complexity►Outputting the routeOutputting the route

ReviewReview

►Memoryless PropertyMemoryless Property►Optimal substructure and Overlapping Optimal substructure and Overlapping

SubproblemsSubproblems►State representation and formula for State representation and formula for

state transferstate transfer►Bottom-up approach VS top-down Bottom-up approach VS top-down

approachapproach►Tradeoff between memory and runtimeTradeoff between memory and runtime

Common ModelsCommon Models

►DP on rectangular arraysDP on rectangular arrays►DP on treesDP on trees►DP on optimized states (ugly states)DP on optimized states (ugly states)

This involves some state representationThis involves some state representation

Further OptimizationFurther Optimization

►Memory optimization – rolling arrayMemory optimization – rolling array►Runtime optimization – reducing the Runtime optimization – reducing the

cost in state transfercost in state transfer

Challenge: The BookcaseChallenge: The Bookcase

►NN books with their height books with their height hhii and and thickness thickness ttii

►You are required to make a bookcase You are required to make a bookcase with three (nonempty) shelveswith three (nonempty) shelves

►You would like to minimize the area of You would like to minimize the area of the bookcase:the bookcase: the sum of the heights of the three shelves the sum of the heights of the three shelves

multiplied by the maximum of the sum of multiplied by the maximum of the sum of thickness in each of the shelvesthickness in each of the shelves

Challenge: Challenge: The BookcaseThe Bookcase

►CoConstraintsnstraints 1 1 ≤ ≤ NN ≤≤ 70 70 150 150 ≤≤ hhii ≤≤ 300 300

5 5 ≤≤ ttii ≤≤ 30 30

►Please do not ask others for the Please do not ask others for the solution. You will waste the problem if solution. You will waste the problem if you do not arrive at the solution you do not arrive at the solution yourself.yourself.

Practice ProblemsPractice Problems

►1058 The Triangle II1058 The Triangle II►3023 Amida De Go II3023 Amida De Go II►3042 Need for Speed3042 Need for Speed►6000 Palindrome6000 Palindrome►6990 Little Shop of Flowers6990 Little Shop of Flowers

ReferencesReferences

1.1. 信息學奧林匹克教程 信息學奧林匹克教程 – – 提高篇 提高篇 吳耀斌吳耀斌 , , 曹利國曹利國 , , 向期中向期中 , , 朱全民編著朱全民編著湖南師范大學出版社湖南師范大學出版社

2.2. 算法藝術與信息學競賽算法藝術與信息學競賽劉汝佳劉汝佳 , , 黃亮著黃亮著清華大學出版社清華大學出版社