Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 · WeakDynamicProgrammingforViscosity Solutions B.Bouchard Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae May2009

Weak Dynamic Programming for ViscositySolutions

B. Bouchard

Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae

May 2009

Joint work with Nizar Touzi, CMAP, Ecole Polytechnique

Motivations

• Consider the control problem in standard form

V (t, x) := supν∈U

J(t, x ; ν) , J(t, x ; ν) := E [f (X νT )|X ν

t = x ]

• To derive the related HJB equation, one uses the DPP

′′V (t, x) = supν∈U

E [V (τ,X ντ )|X ν

t = x ]′′

• Usually not that easy to prove

a Heavy measurable selection argument ?(t, x) 7→ νε(t, x) s.t. J(t, x ; νε(t, x)) ≥ V (t, x)− ε

b Continuity of the value function ?(t, x) ∈ Bri (ti , xi ) 7→ νε(ti , xi ) s.t.

J(t, x ; νε(ti , xi )) ≥ J(ti , xi ; νε(ti , xi ))− ε ≥ V (ti , xi )− 2ε ≥ V (t, x)− 3ε

• Our aim is to provide a weak version, much easier to prove.

Motivations




t = x ]




t = x ]′′






Motivations




t = x ]




t = x ]′′






Motivations




t = x ]




t = x ]′′






Motivations




t = x ]




t = x ]′′






Motivations




t = x ]




t = x ]′′






Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :

U0, a collection of Rd -valued progressively measurable processes.

• Controlled process :

(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0

rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )

.

Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :



(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0

rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )

.

Framework

• (Ω,F ,F := (Ft)t≤T ,P), T > 0.

• Controls :



(τ, ξ; ν) ∈ S × U0 7−→ X ντ,ξ ∈ H0

rcll(Rd )

with [0,T ]× Rd ⊂ S ⊂

(τ, ξ) : τ ∈ T[0,T ] and ξ ∈ L0τ (Rd )

.

Reward and Value functions

• Reward function

J(t, x ; ν) := E[f(X ν

t,x(T ))]

defined for controls ν in

U :=ν ∈ U0 : E

[|f (X ν

t,x(T ))|]<∞ ∀ (t, x) ∈ [0,T ]× Rd

.

• Admissibility : a control ν ∈ U is t-admissible if it isindependent of Ft . We denote by Ut the collection of suchprocesses.

• Value function :

V (t, x) := supν∈Ut

J(t, x ; ν) for (t, x) ∈ [0,T ]× Rd .


• Reward function

J(t, x ; ν) := E[f(X ν

t,x(T ))]


U :=ν ∈ U0 : E

[|f (X ν

t,x(T ))|]<∞ ∀ (t, x) ∈ [0,T ]× Rd

.




J(t, x ; ν) for (t, x) ∈ [0,T ]× Rd .


• Reward function

J(t, x ; ν) := E[f(X ν

t,x(T ))]


U :=ν ∈ U0 : E

[|f (X ν

t,x(T ))|]<∞ ∀ (t, x) ∈ [0,T ]× Rd

.




J(t, x ; ν) for (t, x) ∈ [0,T ]× Rd .

AssumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

A1 (Independence) The process X νt,x is independent of Ft .

A2 (Causality) ∀ ν ∈ Ut : ν = ν on A ⊂ F ⇒ X νt,x = X ν

t,x on A.

A3 (Stability under concatenation) ∀ ν ∈ Ut , θ ∈ T t[t,T ] :

ν1[0,θ] + ν1(θ,T ] ∈ Ut .

A4 (Consistency with deterministic initial data) ∀ θ ∈ T t[t,T ] :

a. For P-a.e ω ∈ Ω, ∃ νω ∈ Uθ(ω) s.t.

E[f(X ν

t,x(T ))|Fθ]

(ω) ≤ J(θ(ω),X νt,x(θ)(ω); νω)

b. ∀ t ≤ s ≤ T , θ ∈ T t[t,s], ν ∈ Us , and ν := ν1[0,θ] + ν1(θ,T ] :

E[f(X ν

t,x(T ))|Fθ]

(ω) = J(θ(ω),X νt,x(θ)(ω); ν) for P− a.e. ω ∈ Ω.




t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



E[f(X ν

t,x(T ))|Fθ]



E[f(X ν

t,x(T ))|Fθ]





t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



E[f(X ν

t,x(T ))|Fθ]



E[f(X ν

t,x(T ))|Fθ]





t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



E[f(X ν

t,x(T ))|Fθ]



E[f(X ν

t,x(T ))|Fθ]





t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



E[f(X ν

t,x(T ))|Fθ]



E[f(X ν

t,x(T ))|Fθ]





t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



E[f(X ν

t,x(T ))|Fθ]



E[f(X ν

t,x(T ))|Fθ]


The case where J(·; ν) is l.s.c. and V is continuous

• Aim : Prove the DPP for τ ∈ T t[t,T ] (independent on Ft)

′′V (t, x) = supν∈Ut

E[V (τ,X ν

t,x(τ))]′′

• Easy inequality : V (t, x) ≤ supν∈Ut E[V (τ,X ν

t,x(τ))]

Proof :

V (t, x) = supν∈Ut

E[E[f (X ν

t,x(T ))|Fτ]]

where for some νω ∈ Uτ(ω)

E[f (X ν

t,x(T ))|Fτ]

(ω) ≤ J(τ(ω),X νt,x(τ)(ω); νω)

≤ V (τ(ω),X νt,x(τ)(ω))




E[V (τ,X ν

t,x(τ))]′′


t,x(τ))]

Proof :


E[E[f (X ν

t,x(T ))|Fτ]]


E[f (X ν

t,x(T ))|Fτ]


≤ V (τ(ω),X νt,x(τ)(ω))




E[V (τ,X ν

t,x(τ))]′′


t,x(τ))]

Proof :


E[E[f (X ν

t,x(T ))|Fτ]]


E[f (X ν

t,x(T ))|Fτ]


≤ V (τ(ω),X νt,x(τ)(ω))




E[V (τ,X ν

t,x(τ))]′′


t,x(τ))]

Proof :


E[E[f (X ν

t,x(T ))|Fτ]]


E[f (X ν

t,x(T ))|Fτ]


≤ V (τ(ω),X νt,x(τ)(ω))




E[V (τ,X ν

t,x(τ))]′′


t,x(τ))]

Proof :


E[E[f (X ν

t,x(T ))|Fτ]]


E[f (X ν

t,x(T ))|Fτ]


≤ V (τ(ω),X νt,x(τ)(ω))




E[V (τ,X ν

t,x(τ))]′′

• Easy inequality :

V (t, x) ≤ supν∈Ut

E[V (τ,X ν

t,x(τ))]

• More difficult one :

V (t, x) ≥ supν∈Ut

E[V (τ,X ν

t,x(τ))]




E[V (τ,X ν

t,x(τ))]′′

• Easy inequality :

V (t, x) ≤ supν∈Ut

E[V (τ,X ν

t,x(τ))]

• More difficult one :


E[V (τ,X ν

t,x(τ))]


Proof : Using Vitali’s covering Lemma, find (ti , xi , ri , νi )i≥1 suchthat,

J(t, x ; ν i ) + ε ≥ J(ti , xi ; νi ) ≥ V (ti , xi )− ε ≥ V (t, x)− 2ε,

on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]

∑i≥1

1Ai (τ,Xνt,x(τ))ν i .




on (ti − ri , ti ]× Bri (xi )

and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]

∑i≥1





on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]

∑i≥1





on (ti − ri , ti ]× Bri (xi ) and also on Ai := (ti − ri , ti ]× Bi , apartition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,τ ]ν + 1(τ,T ]

∑i≥1



Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1

J(τ,X νt,x(τ); ν i )1Ai

(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .


Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1


(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .


Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1


(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .


Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1


(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .


Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1


(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .


Proof : Then,

E[f(X νε

t,x(T ))|Fτ]

=∑i≥1


(τ,X ν

t,x(τ))

≥∑i≥1

(V (τ,X ν

t,x(τ))− 3ε)1Ai

(τ,X ν

t,x(τ))

= V (τ,X νt,x(τ))− 3ε

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fτ]]

≥ E[V(τ,X ν

t,x(τ))]− 3ε .

Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof

: It is in general not difficult to obtain.

• The upper-semicontinuity of V is also very important

: It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

Problems

• The lower-semicontinuity of J(·; ν) is very important in thisproof : It is in general not difficult to obtain.



Problems




Problems


• The upper-semicontinuity of V is also very important : It ismuch more difficult to obtain, especially when controls are notuniformly bounded (singular control).

Observation

To derive the PDE in the viscosity sense, try to obtain :


E[V (τ,X ν

t,x(τ)]

but one only needs :


E[ϕ(τ,X ν

t,x(τ)]

for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

Observation



E[V (τ,X ν

t,x(τ)]



E[ϕ(τ,X ν

t,x(τ)]

for all smooth function such that (t, x) achieves a minimum ofV − ϕ.

ϕ being smooth it should be much easier to prove ! !

Observation



E[V (τ,X ν

t,x(τ)]



E[ϕ(τ,X ν

t,x(τ)]

for all smooth function such that (t, x) achieves a minimum ofV − ϕ.ϕ being smooth it should be much easier to prove ! !

The weak DPPAssume that for all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut

lim inf(t′,x ′)→(t,x), t′≤t

J(t ′, x ′; ν) ≥ J(t, x ; ν).

Theorem : Fix θν , ν ∈ Ut ⊂ T t[t,T ] a family of stopping times.

Then, for any upper-semicontinuous function ϕ such that V ≥ ϕon [t,T ]× Rd , we have

V (t, x) ≥ supν∈Uϕt

E[ϕ(θν ,X ν

t,x(θν))],

where Uϕt =ν ∈ Ut : E

[ϕ(θν ,X ν

t,x(θν))+]<∞ or E

[ϕ(θν ,X ν

t,x(θν))−]<∞

.

Remark : The minimum can be taken to be local if(θν ,X ν

t,x(θν)), ν ∈ Ut is bounded in L∞. In practice ϕ is takento be C 1,2.



J(t ′, x ′; ν) ≥ J(t, x ; ν).




E[ϕ(θν ,X ν

t,x(θν))],


[ϕ(θν ,X ν


[ϕ(θν ,X ν

t,x(θν))−]<∞

.





J(t ′, x ′; ν) ≥ J(t, x ; ν).




E[ϕ(θν ,X ν

t,x(θν))],


[ϕ(θν ,X ν


[ϕ(θν ,X ν

t,x(θν))−]<∞

.


t,x(θν)), ν ∈ Ut is bounded in L∞.

In practice ϕ is takento be C 1,2.



J(t ′, x ′; ν) ≥ J(t, x ; ν).




E[ϕ(θν ,X ν

t,x(θν))],


[ϕ(θν ,X ν


[ϕ(θν ,X ν

t,x(θν))−]<∞

.



The weak DPPProof.

For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that

J(t, x ; ν i )+ε ≥ J(ti , xi ; νi ) ≥ V (ti , xi )−ε ≥ ϕ(ti , xi )−ε ≥ ϕ(t, x)−2ε,

on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1

J(θν ,X νt,x(θν); ν i )1Ai

(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν

t,x(θν))− 3ε)1Ai

(θν ,X ν

t,x(θν))

= ϕ(θν ,X νt,x(θν))− 3ε

The weak DPPProof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that


on Ai := (ti − ri , ti ]× Bi , a partition of [t,T ]× Rd .

Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν


(θν ,X ν

t,x(θν))





νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν


(θν ,X ν

t,x(θν))





νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν


(θν ,X ν

t,x(θν))





νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν


(θν ,X ν

t,x(θν))





νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

Then,

E[f(X νε

t,x(T ))|Fνθ]

=∑i≥1


(θν ,X ν

t,x(θν))

≥∑i≥1

(ϕ(θν ,X ν


(θν ,X ν

t,x(θν))


The weak DPP

Proof. For i ≥ 1, fix ri > 0 and ν i ∈ Uti such that onAi := (ti − ri , ti ]× Bi , disjoint sets that cover [t,T ]× Rd .Given ν ∈ Ut , define

νε := 1[t,θν ]ν + 1(θν ,T ]

∑i≥1

1Ai (θν ,X ν

t,x(θν))ν i .

and

V (t, x) ≥ J(t, x ; νε)

= E[E[f(X νε

t,x(T ))|Fνθ]]

≥ E[ϕ(θν ,X ν

t,x(θν))]− 3ε .

The weak DPP

Using test functions makes the proof straightforward :

supν∈Uϕt

E[ϕ(θν ,X ν

t,x(θν))]≤ V (t, x)

≤ supν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]

Remark : If X νt,x(θν), ν ∈ Ut is bounded in L∞, one can

approximate V∗ from below by smooth functions and obtain :

supν∈Ut

E[V∗(θν ,X ν

t,x(θν))]≤ V (t, x) ≤ sup

ν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]

The weak DPP


supν∈Uϕt

E[ϕ(θν ,X ν


ν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]



supν∈Ut

E[V∗(θν ,X ν


ν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]

The weak DPP


supν∈Uϕt

E[ϕ(θν ,X ν


ν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]



supν∈Ut

E[V∗(θν ,X ν


ν∈Ut

E[V ∗(θν ,X ν

t,x(θν))]

Example : Framework

• Controlled process

dX (r) = µ (X (r), νr ) dr + σ (X (r), νr ) dWr

• U = square integrable progressively measurable processes withvalues in U ⊂ Rd

• f is l.s.c with f − with linear growth, µ and σ Lipschitzcontinuous.

Example : Framework





Example : Framework





Example : Verification of the assumptions

For all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :

L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X ν

t,x(T ) in L2

⇒ lim inf E[f (X ν

t′,x ′(T ))]≥ E

[f (X ν

t,x(T ))].



t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X ν

t,x(T ) in L2


t′,x ′(T ))]≥ E

[f (X ν

t,x(T ))].



t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X ν

t,x(T ) in L2


t′,x ′(T ))]≥ E

[f (X ν

t,x(T ))].



t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .



L.s.c. (t ′, x ′)→ (t, x) ⇒ X νt′,x ′(T )→ X ν

t,x(T ) in L2


t′,x ′(T ))]≥ E

[f (X ν

t,x(T ))].



t,x on A.


ν1[0,θ] + ν1(θ,T ] ∈ Ut .

Example : Verification of the assumptionsFor all (t, x) ∈ [0,T ]× Rd and ν ∈ Ut :



E[f(X ν

t,x(T ))|Fθ]


Proof. Canonical space : W (ω) = ω. Set Ts(ω) := (ωr − ωs)r≥s andωs := (ωr∧s)r≥0.

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„

Xν(ωθ(ω)+Tθ(ω)(ω))

θ(ω),Xνt,x (θ)(ω) (T )(Tθ(ω)(ω))

«dP(Tθ(ω)(ω)) .

=

Zf„



«dP(ω)

= J(θ(ω),X νt,x(θ)(ω); νω)

where, νω(ω) := ν(ωθ(ω) + Tθ(ω)(ω)) ∈ Uθ(ω).




E[f(X ν

t,x(T ))|Fθ]



Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„



«dP(Tθ(ω)(ω)) .

=

Zf„



«dP(ω)






E[f(X ν

t,x(T ))|Fθ]



Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„



«dP(Tθ(ω)(ω)) .

=

Zf„



«dP(ω)






E[f(X ν

t,x(T ))|Fθ]


Proof.

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„



«dP(ω),

and therefore

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf“X ν(Ts (ω))θ(ω),Xνt,x (θ)(ω)(T )(Tθ(ω)(ω))

”dP(ω)

= J(θ(ω),X νt,x(θ)(ω); ν) .




E[f(X ν

t,x(T ))|Fθ]


Proof.

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„



«dP(ω),

and therefore

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =


”dP(ω)

= J(θ(ω),X νt,x(θ)(ω); ν) .




E[f(X ν

t,x(T ))|Fθ]


Proof.

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =

Zf„



«dP(ω),

and therefore

Eˆf`X ν

t,x(T )´|Fθ˜(ω) =


”dP(ω)

= J(θ(ω),X νt,x(θ)(ω); ν) .

Example : Super-solution property

• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X ν

t,x(θν))]


• Want to prove that V∗ is a viscosity super-solution of

infu∈U

(−LuV∗) ≥ 0

• By usingV (t, x) ≥ sup

ν∈UϕtE[ϕ(θν ,X ν

t,x(θν))]


• (t0, x0) ∈ [0,T )× Rd s.t. 0 = (V∗ − ϕ)(t0, x0) = min(V∗ − ϕ)

• Assume that −Luϕ(t0, x0) < 0, for some u ∈ U.

• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.

• Then, −Luϕ(t, x) ≤ 0 on Br := |t − t0| ≤ r , |x − x0| ≤ r.

• Fix (tn, xn,V (tn, xn))→ (t0, x0,V∗(t0, x0)).

• Set θn := infs ≥ tn : (s,X u

tn,xn(s)) /∈ Brso that

V (tn, xn) + ιn = ϕ(tn, xn) ≤ E[ϕ(θn,X u

tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0

, i.e. V (tn, xn) ≤ E[ϕ(θn,X u

tn,xn(θn))]− r4/2.

• While V (tn, xn) ≥ E[ϕ(θn,X u

tn,xn(θn))]by the weak DPP.




• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0


tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X u

tn,xn(θn))]− r4/2.






• Set ϕ(t, x) := ϕ(t, x)− |t − t0|4 − |x − x0|4.






tn,xn(θn))]

≤ E[ϕ(θn,X u

tn,xn(θn))]− r4

for some ιn → 0, i.e. V (tn, xn) ≤ E[ϕ(θn,X u

tn,xn(θn))]− r4/2.



Extensions

• Optimal control with running gain

• Optimal stopping

• Mixed optimal control/stopping, impulse control, ...

Extensions




Extensions




Documents

Weak Dynamic Programming for Viscosity Solutions · 2013-04-16 · WeakDynamicProgrammingforViscosity Solutions B.Bouchard Ceremade, Univ. Paris-Dauphine, and, Crest, Ensae May2009