Monte Carlo 模拟第四章 Monte Carlo 积分. Monte Carlo 法的重要应用领域之一：计算积分和多重积分适用于求解： 1. 被积函数、积分边界复杂，难以用解析方法或一般的数

Monte CarloMonte Carlo 模拟模拟

第四章第四章 Monte CarloMonte Carlo 积积分分

第四章第四章 Monte CarloMonte Carlo 积分积分

Monte Carlo 法的重要应用领域之一：计算积分和多重积分适用于求解：

1. 被积函数、积分边界复杂，难以用解析方法或一般的数值方法求解；

2. 被积函数的具体形式未知，只知道由模拟返回的函数值。

本章内容：

用 Monte Carlo 法求定积分的几种方法：均匀投点法、期望值估计法、重要抽样法、半解析法、…

第四章第四章 Monte CarloMonte Carlo 积分积分目的：计算一个定积分（一维或多维）

dV

g(x) dx I

数值方法：

将积分区间分成 n 个子区间，用一些近似的方法计算各个子区间的积分值，然后对 n 个子区间的积分值求和

• 梯形法 (trapezoidal rule) ：对每个子区间用梯形近似• Simpson’s rule ： approximating the integral of a functi

on g using quadratic polynomials

)]()(4)([3

1)()( 210

0

00

xgxgxghdxxgdxxghx

x

x

x

hxxxx 1201

第四章第四章 Monte CarloMonte Carlo 积分积分

数值方法存在的问题 :

计算速度慢、精度低：

需计算的函数值的数目随着积分维数急剧增长

不恰当的子区间划分将导致不能很好地近似表示被积函数 g(x)导致计算误差

积分维数 d=10, 各方向分点数目 n=50, 需计算函数值的数目： ~nd = 5010

第四章第四章 Monte CarloMonte Carlo 积分积分 Monte Carlo 方法可用于计算任何的 d 重积分

两种方法计算 d- 重积分的误差比较

Simpson’s rule,… dNE /1

21 NE purely statistical,

not rely on the dimension !

Monte Carlo method WINS, when d >> 3

Monte Carlo method

Monte CarloMonte Carlo 模拟模拟第四章第四章 Monte CarloMonte Carlo 积积

分分1. Hit-or-Miss Method

2. Sample Mean Method

3. Variance Reduction: Importance Sampling Method

4. Correlation methods for variance reduction

1. Hit-or-Miss Method1. Hit-or-Miss Method

•Evaluation of a definite integral

dxxgIb

a )(

xxghxg anyfor )(,0)(

a b

hX

X

X

XX

X

O

O

OO

O

OO

•Probability that a random point reside inside the area

N

M

hab

Ip

)(

N

MhabI )(

N : Total number of points

M : points that reside inside the region


Sample uniformly from the rectangular region

[a,b]x[0,h]

a)-h(bI

p

The probability that we are below the curve is

So, if we can estimate p, we can estimate I:

a)-h(bpI ˆˆ

where is our estimate of p

p


We can easily estimate p:

throw N “uniform darts” at the rectangle

NM

pˆ let

let M be the number of times you end up under the curve y=g(x)


a b

hX

X

X

XX

X

O

O

OO

O

OO

Start

Set N : large integer

M = 0

Choose a point x in [a,b]

Choose a point y in [0,h]

if [x,y] reside inside then M = M+1

I = (b-a) h (M/N)

End

LoopN times

auabx 1)(

2huy


Error Analysis of the Hit-or-Miss Method

It is important to know how accurate the result of simulations are

note that M is binomial(M,p) )1()()( 2 PNpMNpME

IpabhMEN

abh

abhN

MEabhpEIE

)()()(

)()(ˆ)ˆ(

N

ppabhM

N

abh

abhN

MabhpI

)1()()(

)(

)()(ˆ)ˆ(

222

2

22

222

N

Mpp ˆ )1(

)()ˆ(

N

MM

N

abhI

21)1(

)()ˆ(

NN

ppabhI


推广到多重积分：

],[)(,0)(

,,,,,,

),,,()(

2121

2121

1

1

2

2

baxxghxg

bbbbaaaa

dxdxdxxxxgxdxgI

dd

b

a

dd

b

a

b

a

b

a

d

d

d ,,, 21 区间 ],[ ba

内均匀分布的随机数=hr : 区间 [0,h] 内均匀分布的随机数

选取 n 组 },{

，若其中满足 g()>的共有 m 组

nii

d

i

b

a

Iabhn

mxdxgI

)()(1


积分结果的置信水平：

对于任意给定的正数，怎样才能保证积分计算值 In 与真值 I 之差的绝对值小于的概率大于 (0< <1)

||)(|| pn

mabhIIn

)1()(

})1(

||{}||)({)(

pp

n

abht

tpnp

npmPp

n

mabhPP

根据中心极限定理

nN

pnp

npm)1,0(~

)1(

||

1)(2)()()( tttP )ˆ()1(

)( Itn

ppabht






2. Sample Mean Method2. Sample Mean Method

dV

xdxgI

)(

)]([

1)()(

XgEV

xdV

xgVxdxg I

d

Vd

dV dd

设欲求的 d- 重积分为

令 X 为积分域 Vd 上均匀分布的随机向量，其概率密度函数为：

dVxf

1)(


产生容量为 n 的 X 的随机样本 Xi, 并计算 g(Xi), 则根据大数定理，当 n 足够大时

n

iiXg

n XgE

1

)(1

)]([

)]([ XgEVI d

X 在积分域 Vd 上均匀分布

n

ii

dn Xg

n

V II

1

)(


误差分析：oThis estimator is “unbiased”:

IxdxgxdV

xgn

V

XgEn

VXg

n

V E IE

dd V

n

iV

d

d

n

ii

dn

ii

dn

)(1

)(

)]([)(][

1

11


oVariance of this estimator:

hVd

d

Vd

di

d

n

ii

dn

ii

dnI

Vn

xdV

IxgV

n

xdV

xgExgn

VXgV

n

V

XgVn

VXg

n

V V IV

d

d

n

1)(

1

1)]([)()]([

)]([)(][

2

222

12

2

1

2

222222

2

)]([)]([

)(

ndd

Vd

dh

IXgEVIXgEV

xdV

IxgVV

d


一维积分的情况：

)]([)(

1)()()(

XgEab

dxab

xgabdxxg I b

a

b

a

n

iin Xg

n

ab XgEabII

1

)()(

)]([ˆ)(

2222 )]([)(11

nhI IXgEabn

Vnn


Start


s = 0

xn = (b-a) un + a

yn = g(xn)

s = s + yn

Estimate mean In=s/N

End

LoopN times


dx e I3

0

x

Example:

(we know that the answer is e3-1 19.08554)

write this as

]3E[e dx 31

e3 I X3

0

x

where X~unif(0,3)


write this as

]3E[e dx 31

e3 I X3

0

x

where X~unif(0,3)

estimate this with

n

1i

xX ien1

3][eE3 I ˆˆ

where X1, X2, …, Xn are n independent unif(0,3)’s.


Simulation Results: true = 19.08554,

n=100,000

1 19.10724

I

2 19.08260

3 18.97227

4 19.06814

5 19.13261

Simulation


Comparison of Hit-and-Miss and Sample Mean Monte Carlo

Let be the hit-and-miss estimator of IHMI

)IVar( )IVar( SMHMˆˆ

Then

Let be the sample mean estimator of ISMI


Comparison of Hit-and-Miss and Sample Mean Monte Carlo

Sample mean Monte Carlo is generally preferred over Hit-and-Miss Monte Carlo because:

the estimator from SMMC has lower variance

SMMC does not require a non-negative integrand (or adjustments)

H&M MC requires that you be able to put g(x) in a “box”, so you need to figure out the max value of g(x) over [a,b] and you need to be integrating over a finite integral.






3. Variance reduction: Importance Sampling 3. Variance reduction: Importance Sampling MethodMethod

hI Vnn

12

222222

2

)]([)]([

)(

ndd

Vd

dh

IXgEVIXgEV

xdV

IxgVV

d

Sample mean method:

减小积分误差的方法：• 增大抽样的次数 n ；• 减小方差 Vh


Reducing error

*100 samples reduces the error order of 10

Reducing variance Variance Reduction Technique

The value of variance is closely related to how samples are taken

Unbiased sampling

Biased sampling• More points are taken in important parts of the population


If we are using sample-mean Monte Carlo Method

•Variance depends very much on the behavior of g(x)

g(x) varies little variance is small

g(x) = const variance=0

Evaluation of a integral

N

nnn xg

N

abI

1

)(

Near minimum points contribute less to the summation

Near maximum points contribute more to the summation

More points are sampled near the peak ”importance sampling strategy”


Importance sampling methodImportance sampling method

Basic idea

Put more points near maximum

Put less points near minimumc

g(x)

f(x) b

adxxg I )(

X 的概率密度函数为 f(x)


dV

xdxgI

)(

令 f(x) 为积分域 Vd 上随机向量 X 的概率密度函数

)]([

)()()()(

)()(

XhE

xdxfxhxdxfxf

xgxdxg I

ddd VVV

设欲求的 d- 重积分为

dVVxxfxdxf

d

0)(1)(

)(

)()(

Xf

XgXh


So, we will estimate I by estimating E[h(X)] with

n

ii

n

iin h

nXh

n XhEII

11

1)(

1)]([ˆ

where X1, X2, …, Xn is a random sample from the f(X) distribution.

)(~])(

)([)]([ XfX

Xf

XgEXhEI

3. Variance reduction:Importance Sample 3. Variance reduction:Importance Sample MethodMethod

误差分析：oThis estimator is “unbiased”:

Ixdxgxdxfxhn

XhEn

Xhn

E IE

dd V

n

iV

n

ii

n

iin

)()()(1

)]([1

)(1

][

1

11


oVariance of this estimator:

hV

Vi

n

ii

n

iinI

Vn

xdxfIxhn

xdxfxhExhn

XhVn

XhVn

Xhn

V IV

d

d

n

1)()(

1

)()]([)(1

)]([1

)]([1

)(1

][

2

2

12

1

2

2222

2

)]([)]([

)()(

n

d

IXhEIXhE

x dxfIxhVVh


一维积分的情况：

)]([)()(

)()(

)()(

XhEdxxfxh

dxxfxf

xgdxxg I

b

a

b

a

b

a

n

ii

n

iin h

nXh

n XhEII

11

1)(

1)]([ˆ

222 )]([11

nn

IXhEn

Vn hI


Start


s = 0

Generate xn according to f(x)

hn = g (xn) / f ( xn)

Add hn to s

I =s1/N

End

LoopN times






4. Correlation methods for Variance 4. Correlation methods for Variance reductionreduction

Correlation method:

Use correlated points in the sampling to reduce the variance of the integrand and improve the efficiency of the estimation

•Control variates•Antithetic variables


Control variates

ddd VVV

xdxfxgxdxfxdxgI

)]()([)()(

f(x): Control variate for g(x), must satisfy:

•Simple enough to allow analytical integration •Should mimic g(x) to absorb most of its fluctuation

f(x) ~ g(x)


第九章第九章 Monte CarloMonte Carlo 积分积分

Hit-or-Miss Method

Sample Mean Method

Variance Reduction Technique

Variance Reduction using Rejection Technique

Importance Sampling Method


IntroductionMonte Carlo Method and Sampling Distribution

Monte Carlo Method : Take values from random sample From central limit theorem,

3 rule

Most probable error

Important characteristics

N/ 22

9973.0)33( XP

NError

6745.0

NError /1 Error


IntroductionReducing error

*100 samples reduces the error order of 10 Reducing variance Variance Reduction Technique

The value of variance is closely related to how samples are taken

Unbiased samplingBiased sampling

More points are taken in important parts of the population

Variance Reduction TechniqueMotivation

If we are using sample-mean Monte Carlo MethodVariance depends very much on the behavior of (x)

(x) varies little variance is small(x) = const variance=0

Evaluation of a integral

Near minimum points contribute less to the summationNear maximum points contribute more to the summation

More points are sampled near the peak ”importance sampling strategy”

N

nnY

xN

ababI

1

)()('


n

1ii )g(X

n1

a)-(b I

X1, X2, …, Xn iid -> g(X1), g(X2), …, g(Xn) iid

Let Yi=g(Xi) for i=1,2,…,n

Then

Y a)-(b I ˆ

and we can once again invoke the CLT.


)N(I, I 2I

ˆ

For n “large enough” (n>30),

So, a confidence interval for I is roughly given by

I/2 z I ˆˆ

but since we don’t know , we’ll have to be content with the further approximation:

I

I/2 s z I ˆˆ


b

a

2x1/2 dx e x: I

By the way…

No one ever said that you have to use the uniform distribution

Example:

0

b][a,2x1/2 dx (x)I 2e x

21

(X)I XE21

b][a,1/2

where X~exp(rate=2).

第九章第九章 Monte CarloMonte Carlo 积分积分

1.2 Hit-or-Miss Method

1.3 Sample Mean Method

2.1 Variance Reduction Technique

2.3 Variance Reduction using Rejection Technique

2.4 Importance Sampling Method


Variance Reduction for Hit-or-Miss method

•In the domain [a,b] choose a comparison function

dxxwA

dxxwxW

xxw

b

a

x

)(

)()(

)()(

)(xwAu )(1 AuWx

a b

w(x)

X

X

XX

X

O

O

OO

O

OO

(x)

Points are generated on the area under w(x) function

Random variable that follows distribution w(x)


Points lying above (x) is rejected

N

NAI

'

nnn uxwy )(

)( if 0

)( if 1

nn

nnn xy

xyq

q 1 0P(q) r 1-r

AIr /

a b

w(x)

X

X

XX

X

O

O

OO

O

OO

(x)


Error Analysis)1()( ,)( rrQVrQE

)(QAEArI

N

IAII RJ

error

)(67.0

Hit or Miss method

Error reduction

habA )(

0Error then IA


Start


N’ = 0

Generate u1, x= W-1(Au1)

Generate u2, y=u2 w(x)

If y<= f(x) accept value N’ = N’+1Else : reject value

I = (b-a) h (N’/N)

End

LoopN times

N

IAII RJ

error

)'('67.0

a b

w(x)

X

X

XX

X

O

O

OO

O

OO

(x)

Documents

Monte Carlo 模拟 第四章 Monte Carlo 积分. Monte Carlo 法的重要应用领域之一：计算积分和多重积分 适用于求解： 1. 被积函数、积分边界复杂，难以用解析方法或一般的数

Monte Carlo 模拟第四章 Monte Carlo 积分. Monte Carlo 法的重要应用领域之一：计算积分和多重积分适用于求解： 1. 被积函数、积分边界复杂，难以用解析方法或一般的数