Upload
others
View
14
Download
0
Embed Size (px)
Citation preview
Mathematical EconomicsDiscrete time: calculus of variations problem
Paulo Brito
[email protected] of Lisbon
November 23, 2018
Discrete time: intuitionAssume we have a cake of initial size W0 = ϕ and we want to eatit completely by time t = T, then WT = 0The size of the cake evolution is:
Wt+1 = Wt − Ct ⇒ Wt = W0 −t−1∑s=0
Cs
By imposing the two constraints W0 = ϕ and WT = 0 we have
T−1∑s=0
Ct = ϕ
There is an infinite number of paths {C0, . . .CT−1} verifyingthis conditionExample: if we chose Ct = C̄ constant for all t we get C̄ = ϕ
T .Is this optimal ? Depends on the value functional
Discrete time dynamic optimization: Intuition
A dynamic model is defined over sequences x (or (x, u)) inwhich there is some form of intertemporal interaction:actions today have an impact in the future;Types of time interactions:- intratemporal: iinteraction within one period- intertemporal: interaction across periodsAdmissible sequences: the set X contains a large number(possibly infinite) of admissible paths verifying a givenintratemporal relation together with some informationregarding initial and/or terminal data ;Optimal sequences: an intertemporal optimalitycriterium allows for choosing the best admissible sequence.
The discrete time calculus of variations problem (CV)
Calculus of variations problem: find x = {xt}Tt=0 ∈ X , i.e., a path
belonging to the set of admissible paths X , that maximises anintertemporal objective function (or value functional)
J(x) =T−1∑t=0
F(xt+1, xt, t), x ∈ X
where F(t, xt, xt+1) specifies an intratemporal relation for the valueof the action upon the state variable, within period t (i.e., betweentimes t and t + 1).
time
period
state
flow
0
x0
1
x1
2
x2
t
xt
t+1
xt+1
T−2
xT−2
T−1
xT−1
T
xT
0
F0
1
F1
t
Ft
T−2
FT−2
T−1
FT−1
The value for changing the state variable in period t is
Ft = F(xt+1, xt, t).
The value of the a given sequence of actions across T − 1periods is
J(x) = F(0, x0, x1) + . . .+ F(t, xt, xt+1) + . . .+ F(T − 1, xT−1, xT)
The optimal sequence x∗ has value
J∗ = J(x∗) = maxx
{J(x) : x ∈ X}
The discrete time optimal control problem (OC)
Optimal control problem: find the pair(x, u) = ({xt}T
t=0, {ut}T−1t=0 ) ∈ D = X × U where D is defined by
the sequence of intratemporal relations for periods 0 to T − 1
xt+1 = G(xt, ut, t), t = 0, 1, . . . , t, . . .T − 1
plus other conditions, that maximises an intertemporal objectivefunctional
J(x, u) =T−1∑t=0
F(t, xt, ut)
Observation: J(x) has terminal time T − 1 because if we knowthe optimal x∗T−1 and u∗
T−1, we know the optimal x∗T, becausex∗T = G(x∗T−1, u∗
T−1,T − 1).
time
period
state
controlvalue flow
0
x0
1
x1
2
x2
t
xt
t+1
xt+1
T−2
xT−2
T−1
xT−1
T
xT
0
u0
1
u1
t
ut
T−2
uT−2
T−1
uT−1
0
F0
1
F1
t
Ft
T−2
FT−2
T−1
FT−1
The value associated to choosing the control ut, given the statext, throughout period t is
Ft = F(xt, ut, t). Choosing the control ut, generates a the value for the state variableat the end of period t, xt+1 = g(xt, ut, t)The value of a given sequence of controls across T − 1periods is
J(u, x) = F(0, x0, u0) + . . .+ F(t, xt, ut) + . . .+ F(T − 1, xT−1, uT−1)
The optimal sequence u∗ has valueJ∗ = J(u∗) = max
u{J(x, u) : (x, u) ∈ D}
Calculus of variations problems
We consider the following problems:Simplest problem: X = {T, x0, and xT given}Free terminal state problem: X = {x0, and T given}, xT freeConstrained terminal state problem:X = {x0, and T, and h(xT) ≥ 0 given}Discounted infinite horizon problems: T = ∞ andlimt→∞ xt free or constrained
Calculus of variations: simplest problemThe problem : Find x∗ = {x∗t }T
t=0 that maximizes
J(x) =T−1∑t=0
F(xt+1, xt, t), s. t. x0 = ϕ0, xT = ϕT (1)
where T, ϕ0 and ϕT are known.First order necessary conditions
Propositionif x∗ ≡ {x∗0, x∗1, . . . , x∗T} is a solution of problem (1) , it verifies theEuler-Lagrange equation and the admissibility conditions
∂F∂xt
(x∗t , x∗t−1, t − 1) + ∂F∂xt
(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1x∗0 = ϕ0, t = 0x∗T = ϕT, t = T
Proof .
Application: cake eating problemThe problem
maxC
{ T∑t=0
βt ln(Ct) : Wt+1 = Wt − Ct, W0 = ϕ, WT = 0}
Assumptions: 0 < β < 1, T ≥ 1, ϕ > 0Intuition:
at time t = 0, we have a cake of initial size ϕ, and we want toconsume it completely until time T (known)by choosing a sequence of bites such that:(1) we value independently each bite (the value functional is asum);(2) the instantaneous pleasure of each bite is increasing with thesize of the bite, but at a decreasing rate (the utility functionu(Ct) is concave);(3) we are impatient: when we plan to eat the cake we value morethe immediate bites rather than future bites ( there is a discountfactor βt which decreases with time).How should we eat the cake ?
Application: cake eating as a CV problem
We can transform the previous problem into a calculus ofvariations problem
maxW
{ T∑t=0
βt ln(Wt − Wt+1) : W0 = ϕ, WT = 0}
Application: cake eating problem - solutionThe optimal cake size is W∗ = {ϕ, . . . ,W∗
t , . . . ,W∗T−1, 0} where
W∗t =
(βt − βT
1 − βT
)ϕ, t = 0, 1, . . .T
It slightly bends down from a linear path because of time discounting,as 0 < β < 1 Proof .
Calculus of variations: free terminal state problemFind x∗ = {x∗t , }T
t=0 that maximizes
J(x) =T−1∑t=0
F(xt+1, xt, t) s.t. x0 = ϕ0 (2)
where ϕ0 and T is given and xT is free.First order necessary conditions:
Propositionif x∗ is a solution of problem (2) , it verifies the Euler-Lagrangeequation the admissibility condition and the transversality condition
∂F∂xt
(x∗t , x∗t−1, t − 1) + ∂F∂xt
(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1
x∗0 = ϕ0, t = 0∂F∂xT
(x∗T, x∗T−1,T − 1) = 0, t = T
Proof .
Cake eating problem: free terminal state
Problem
max{C}
T∑t=0
βt ln(Ct), subject to Wt+1 = Wt − Ct, W0 = ϕ, WTfree
The problem is ill-posed: there is no solution with economicmeaning Proof .Reason: the transversality condition only holds if C0 = ∞this is intuitive: if we had no restriction on the terminal size ofthe case (or if could borrow freely) we would overeat.We have to redefine the problem to obtain a reasonable solution.
Calculus of variations: constrained terminal stateFind x∗ = {x∗t }T
t=0 that maximizes
J(x) =T−1∑t=0
F(xt+1, xt, t) s.t. x0 = ϕ0, xT ≥ ϕT (3)
where T, ϕ0 and ϕT are given.First order necessary conditions:
Propositionif x∗ is a solution of problem (3), it verifies
∂F∂xt
(x∗t , x∗t−1, t − 1) + ∂F∂xt
(x∗t+1, x∗t , t) = 0, t = 1, 2, . . . ,T − 1
x∗0 = ϕ0, t = 0∂F∂xT
(x∗T, x∗T−1,T − 1) · (ϕT − x∗T) = 0, t = T
Proof .
Cake eating problem: constrained terminal state
Problem
max{C}
T∑t=0
βt ln(Ct), subject to Wt+1 = Wt − Ct, W0 = ϕ, WT ≥ 0
F.o.c W∗
t+2 = (1 + β)W∗t+1 − βW∗
t , t = 0, 1, . . . ,T − 2W∗
0 = ϕ, t = 0βT−1
W∗T − W∗
T−1W∗
T = 0, t = T
has the same formal solution as the simplest problem: W∗
T = 0 isdetermined endogenously not by assumption. Proof .
Calculus of variations: discounted infinite horizon 1
The problem: find the infinite sequence x∗ = {x∗t }∞t=0
maxx
∞∑t=0
βtf(xt+1, xt), s.t. x0 = ϕ0 (4)
where, 0 < β < 1, and ϕ0 are given and the terminal state is free(i.e., limt→∞ xt is free).The first order conditions are:
∂f∂xt
(x∗t , x∗t−1) + β∂f∂xt
(xt+1, xt) = 0, t = 0, 1, . . .
x∗0 = x0, t = 0limt→∞ βt−1 ∂f(x∗
t ,x∗t−1)
∂xt= 0, t → ∞
Calculus of variations: discounted infinite horizon 2
The problem: find the infinite sequence x∗ = {x∗t }∞t=0
maxx
∞∑t=0
βtf(xt+1, xt), s.t. x0 = ϕ0, limt→∞
xt ≥ 0 (5)
given β and ϕ0
F.o.c. ∂f∂xt
(x∗t , x∗t−1) + β∂f∂xt
(xt+1, xt) = 0, t = 0, 1, . . .
x∗0 = x0,
limt→∞ βt−1 ∂f∂xt
(x∗t , x∗t−1) · x∗t = 0,
Cake eating problem: infinite horizonProve that if the terminal state is free there is no finite solutionProve that with the terminal condition limt→∞ Wt ≥ 0 thesolution is generated by
W∗t = ϕ0β
t, t = 0, 1, . . . ,∞
Terminal conditions
Problem Given Optimality contitionsT xT T∗ x∗T
(CV1) fixed fixed T xT
(CV2) fixed free T∂F(x∗T, x∗T−1,T − 1)
∂xT= 0
(CV3) fixed xT ≥ ϕT T∂F(x∗T, x∗T−1,T − 1)
∂xT(ϕT − x∗T) = 0
(CV4) ∞ free ∞ limt→∞ βt−1 ∂F(x∗t , x∗t−1)
∂xt= 0
(CV5) ∞ x∞ ≥ 0 ∞ limt→∞ βt−1 ∂F(x∗t , x∗t−1)
∂xtx∗t = 0
Proofs
Proof of proposition 1Let x∗ = {x∗t }T
t=0 be an optimal path. Then the optimal value is
J(x∗) =T−1∑t=0
F(x∗t+1, x∗t , t)
Introduce an admissible perturbation xt = x∗t + εt where ε0 = εT = 0, andεt ̸= 0 for any t ∈ {1, . . . ,T − 1}. The value for the perturbed path is
J(x) =T−1∑t=0
F(x∗t+1 + εt+1, x∗t + εt, t)
Using a Taylor expansion we get first variation in the value functional
J(x)− J(x∗) = ∂F(x∗1, x∗0, 0)∂x0
ε0 +
T−1∑t=1
(∂F(x∗t , x∗t−1, t − 1)
∂xt+
∂F(x∗t+1, x∗t , t)∂xt
)εt+
+∂F(x∗T, x∗T−1,T − 1)
∂xTεT
=
T−1∑t=1
(∂F(x∗t , x∗t−1, t − 1)
∂xt+
∂F(x∗t+1, x∗t , t)∂xt
)εt
The path x is optimal if J(x)− J(x∗) = 0. Then the EL is obtained.Return .
Solution of the cake eating problem 1The Euler-Lagrange equation
Observe that in this problem the utility function, evaluated at theoptimum, is
F∗t = F(W∗
t+1,W∗t , t) = βt ln(W∗
t − W∗t+1)
and the Euler-Lagrange condition is∂F∗
t−1∂Wt
+∂F∗
t∂Wt
= 0
applying to our utility function we have
∂
∂Wt
(βt−1 ln(W∗
t−1 − W∗t ))
+∂
∂Wt
(βt ln(W∗
t − W∗t+1)
)=
= − βt−1
W∗t−1 − W∗
t+
βt
W∗t − W∗
t+1= 0 (6)
is a linear second-order difference equation
W∗t+1 − W∗
t − β(W∗t − W∗
t−1), t = −1, . . . ,T − 3
Solution of cake eating problem 1
If W∗ is a solution of the problem then it verifies the first orderconditions
W∗t+2 = (1 + β)W∗
t+1 − βW∗t , t = 0, . . .T − 2
W∗0 = ϕ
W∗T = 0
To find the solution either we transform to a planar system or are ableto transform to a simpler problem (we follow the second strategy)
Solution of cake eating problem 1Solving the problem: recursive approach
Because Wt+1 − Wt = −Ct, the EL equation is equivalent to
Ct+1 = βCt
This is a first-order DE which has solution
Ct = C0βt, where C0 unknown
Therefore
Wt+1 = Wt − C0βt, for t = 0, 1, . . . ,T − 1
This is a unit-root equation with solution
Wt = ϕ− C0
t−1∑s=0
βs = ϕ− C0
(1 − βt
1 − β
) because W0 = ϕ.
Solution of cake eating problem 1
However, to be optimal, Wt should be admissible, which meanssatisfying {
W∗0 = ϕ
W∗T = 0
But Wt|t=0 = ϕ = ϕ
Wt|t=T = ϕ− C0
(1 − βT
1 − β
)
Setting Wt|t=T = 0 yields C∗0 = ϕ
(1 − β
1 − βT
)Then the solution to the cake eating problem is generated by
W∗t = ϕ
(1 − (1 − β)
(1 − βT)
(1 − βt)
(1 − β)
)= ϕ
(βt − βT
1 − βT
)
Return .
Proof of proposition 2
Using the same method of the proof of Proposition 1 we have
J(x)− J(x∗) = ∂F(x∗1, x∗0, 0)∂x0
ε0 +
T−1∑t=1
(∂F(x∗t , x∗t−1, t − 1)
∂xt+
∂F(x∗t+1, x∗t , t)∂xt
)εt+
+∂F(x∗T, x∗T−1,T − 1)
∂xTεT
=
T−1∑t=1
(∂F(x∗t , x∗t−1, t − 1)
∂xt+
∂F(x∗t+1, x∗t , t)∂xt
)εt+
+∂F(x∗T, x∗T−1,T − 1)
∂xTεT
because the admissibility condition only implies that ε0 = 0. Return .
Cake eating problem: free terminal stateThe first-order conditions are
W∗t+2 = (1 + β)W∗
t+1 − βW∗t , t = 0, 1, . . . ,T − 1
W0 = ϕ, t = 0βT−1
W∗T−W∗
T−1= 0. t = T
Using the same transformation as in the fixed-terminal state problemwe have {
Ct+1 = βCt, t = 0, 1, . . . ,T − 1βT−1
CT−1= 0. t = T
But we already know that the solution to the first equation isCt = C0β
t. This implies that the transversality constraint becomes
βT−1
CT−1=
βT−1
C0βT−1 =1
C0
which can only be zero if C0 = ∞. This means that WT → −∞which does not make sense. The problem is ill-posed.
Return .
Proof of proposition 3
Using the same method of the proof of Proposition 3 but introducing theterminal constraint
J(x) =T−1∑t=0
F(x∗t+1 + εt+1, x∗t + εt, t) + λ(ϕT − x∗T − εT)
where λ is a Lagrange multiplier. Then the variation of the value of theproblem is
J(x)− J(x∗) =T−1∑t=1
(∂F(x∗t , x∗t−1, t − 1)
∂xt+
∂F(x∗t+1, x∗t , t)∂xt
)εt+
+∂F(x∗T, x∗T−1,T − 1)
∂xTεT + λ(ϕT − x∗T − εT)
Proof of proposition 3
Then J(x) = J(x∗) if the EL equation holds,(∂F(x∗T, x∗T−1,T − 1)
∂xT− λ
)εT = 0
where εT is arbitrary, and the Karush-Kuhn-Tucker conditions hold
λ(ϕT − x∗t ) = 0, λ ≥ 0, x∗t ≥ ϕT.
Therefore ∂F∗T−1
∂xT− λ = 0 but
∂F∗T−1
∂xT− λ =
(∂F∗
T−1∂xT
− λ
)(ϕT − x∗t ) =
∂F∗T−1
∂xT(ϕT − x∗t ) = 0
which is the transversality condition. Return .
Solution to the cake eating problem with constrainedterminal state
Now the transversality condition is
−WTβT−1
WT − WT−1= 0
Using our previous transformation we have
−WTβT−1
WT − WT−1= −WT
βT−1
CT−1= −WT
βT−1
βT−1C0= −WT
C0= 0
only if WT = 0 for any finite and positive C0. Return .