12 Ordinary Differential Equations: Initial-Value Problems 2005. 06. 01 자연어처리 연구실 김혜겸 윤도상 최성원

1212Ordinary DifferentialOrdinary Differential

Equations:Equations:Initial-Value ProblemsInitial-Value Problems

2005. 06. 01

자연어처리 연구실김혜겸윤도상최성원

2

IntroductionIntroduction Techniques for solving first-order ordinary differential equations a

re the subject of this chapter– the differential equation is written in the form with the value of functi

on given at ,i.e., .

– Basic idea is to divide the internal of interest into discrete steps and find approxima

tions to the function y at those values of .

Taylor Method

Runge-Kutta Method

Multistep Method

),(' yxfy )(xy 00)( yxy

x

0x

3

12.1 TAYLOR METHODS12.1 TAYLOR METHODS Consider two methods that are based on the Taylor

series representation of unknown function .– Euler’s method

• Uses only the first term in the Taylor series

– Higher order Taylor method

)(xy

4

12.1 TAYLOR METHODS 12.1 TAYLOR METHODS 12.1.1 Euler’s Method12.1.1 Euler’s Method

The simplest method of approximating the solution of the

differential equation

– Start by treating the function as a constant, , and replace the

derivative by the forward difference quotient. This gives

or ( where )

Geometrically, this corresponds to using the line tangent

to the true solution curve to find the value of .

),(' yxfy

),( 00 yxf

'y

))(,( 010001 xxyxfyy ),( 0001 yxhfyy nabh /)(

)(xy 1y

5

12.1 TAYLOR METHODS 12.1 TAYLOR METHODS 12.1.1 Euler’s Method12.1.1 Euler’s Method

Euler Method gives

, for

For example,

),( 111 iiii yxhfyy ni ,...,1

2/1

,2

,0

,),(

0

0

h

y

x

yxyxf

6

Example 12.1 Solving a Simple ODE Example 12.1 Solving a Simple ODE with Euler’s Method(1/2)with Euler’s Method(1/2)

, (i.e., a=0, b=1) , .

First, find the approximate solution for h =0.5 (n =2)

Approximation at is

Second, find the approximate solution at :

yxy ' 10 x 2)0( y

5.01 x

0.3)0.20.0(5.00.2)( 0001 yxhyy

2y 0.120.02 hx

75.4)0.35.0(5.00.3)( 1112 yxhyy

7

Example 12.1 Solving a Simple ODE Example 12.1 Solving a Simple ODE with Euler’s Method(2/2)with Euler’s Method(2/2)

To find a better approximate solution(use n=10 intervals)0.1,...,4.0,3.0,2.0,1.0,0.0 1043210 xxxxxx

8

MATLAB Function For Euler’s Method MATLAB Function For Euler’s Method MATLAB Function For Euler’s Method

9

Example 12.2 Another Example of Example 12.2 Another Example of the Use of Euler’s Methodthe Use of Euler’s Method

Consider the differential equation

– Interval with initial value

– With intervals, the step size

– Exact solution is

),(' yxfy

),1

2(x

xy

,1

0x

0x

20 x 0.0)0( y

10n 2.0/)( nabh)exp( 2xxy

y1=y0+hf(x0,y0) =0+0.2(1)=0.2

y2=y1+hf(x1,y1) =0.2+0.2(0.2(-0.4+1/2))=0.384

.

.

.

10

DiscussionDiscussion How well a numerical technique for solving an initial-

value ordinary differential equation works– Total truncation error for Euler’s method is

– Related to the truncation error of the method is

Result of basic Euler’s method can be improved by using

it with Richardson extrapolation

L

xxLN

hn

1))(exp(

2

0

)(hO

11

j

jjii hcyy

12

)()1(2)(

11

m

mmm

mkYkY

kY

:0

:

:

Y

m

k Step size

Extrapolation level

Computed directly from Euler’s method

11

12.1 TAYLOR METHODS 12.1 TAYLOR METHODS 12.1.2 Higher Order Taylor Method12.1.2 Higher Order Taylor Method

Use more terms in the Taylor series for y, in order to obtain a

higher order truncation error

– However do not have a formula for

– If is not too complicated

• Differentiate with respect to to find a representation for

)()(''2

)(')()( 32

hOxyh

xhyxyyxy

)('' xy),()(' yxfxy

f x )('' xy

12

Example 12.3 Solving a Simple ODE Example 12.3 Solving a Simple ODE with Taylor’s Methodwith Taylor’s Method

Consider the differential equation

with initial condition

To apply the second order Taylor method

This gives the approximation formula

or

For , we find

,' yxy ,10 x 2)0( y

yxyyxdx

dy 1'1)(''

)(''2

)(')()(2

xyh

xhyxyhxy

)1(2

)(2

1 iiiiii yxh

yxhyy

)5.0(2 hn

,8

27

8

201

2

202)1(

2)( 00

2

0001 yxh

yxhyy

.9219.5)8

27

2

11(

8

1)

8

27

2

1(

2

1

8

27)1(

2)( 11

2

1112 yxh

yxhyy

13

MATLAB Function For Taylor’s MethodMATLAB Function For Taylor’s Method MATLAB Function For Taylor’s Method

14

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 배경

– Talyor 해법은 Euler 해법에 비해서 정확하지만 , f(x,y) 의 고계도함수를 구해야 하는 어려움

– Runge-kutta 해법은 talyor 해법의 장점이 되는 계산 정밀도는 유지하되 , 단점을 보완한 방법

15

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.1 Midpoint 12.2.1 Midpoint

MethodMethod

– Runge-kutta 의 해법 중 가장 간단함

– 공식

• Runge-kutta 해법은 더 쉬운 계산법을 사용하여 Tailer 방법을 근사시키는 전략을 사용함

21

12

1

2

1

2

1

kyy

)kh, y hf(x k

), y hf(x k

ii

ii

ii

16


MethodMethod

Example 12.4 Solving a Simple ODE with the Midpoint Method

에서 미분 방정식은 , y(0) = 2yxy '10 x

1) h = 0.5 (n=2) x1 = 0.5

375.3375.12

375.1)5.025.0(5.02

1

2

1

0.1)20(5.0)(5.0),(

201

001002

00001

kyy

yx)kh, y hf(x k

yxyx hf k

2) x2 = 0.0 + 2h = 1.0

922.5547.2375.3

547.2)97.0375.325.05.0(5.02

1

2

1

9375.1)375.35.0(5.0)(5.0),(

212

1112

11111

kyy

)kh, y hf(x K

yxyx hf K

<P406 참조 > Exact : 3.4462 Euler’s : 3.3315

<P406 참조 > Exact : 6.1548 Euler’s : 5.7812

17


MethodMethod

MATLAB Function for the Midpoint MethodFunction [x, y] = RK2 (f, tspan, y0, n)% function [x, y] = RK2 (f, y0, a, b, n)% solve y’ = f(x, y)% with initial condition y(a) = y0% using n steps of the midpoint (RK2) method;a = trans(1); b = trans(2); h = (b - a) / nx = (a+h: h : b);k1 = h * feval (f, a, y0);k2 = h * feval (f, a + h / 2, y0 + k1 / 2);y(1) = y0 + k2;for i = 1 : n-1 k1 = h * feval (f, x(i), y(i) ); k2 = h * feval (f, x(i) + h / 2, y(i) + k1 / 2); y(i+1) = y(i) + k2;endx = [ a x ];y = [ y0 y];

18

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.2 Other Second-Order Runge-Kutta Mehod 12.2.2 Other Second-Order Runge-Kutta Mehod

ss

2 차 runge-kutta 해법의 일반적인 형태

– Modified Euler’s method

– Heun’s method

– Midpoint Method

22111

12122

1

,

kwkwyy

)ka yhc hf(x k

), y hf(x k

nn

nnn

nn

c2 a21

w1 w2

211

12

1

2

1

2

1

,

kkyy

)k yh hf(x k

), y hf(x k

nn

nnn

nn

1 1

1/2 1/2

211

12

1

4

3

4

13

2,

3

2

kkyy

)k yh hf(x k

), y hf(x k

nn

nnn

nn

2/3 2/3

1/4 3/4

21

12

1

2

1,

2

1

kyy

)k yh hf(x k

), y hf(x k

nn

nnn

nn

1/2 1/2

0 1

19

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.3 Third-Order Runge-Kutta Mehods 12.2.3 Third-Order Runge-Kutta Mehods

N=3 이며 , 식은 2 차 방법과 유사하게 기술 됨

3322111

22113133

12122

1

,

,

kwkwkwyy

)kaka yhc hf(x k

)ka yhc hf(x k

), y hf(x k

nn

nn

nn

nn

c3 a31

w1 w2

c2 a21

a32

w3

2/3 0

2/8 3/8

2/3 2/32/3

3/8

1 -1

1/6 4/6

1/2 1/22

1/6

- Nystrom

- Classical

3/4 0

2/9 3/9

1/2 1/23/4

4/9

2/3 0

1/4 0

1/3 1/32/3

3/4

- Nearly Optimal

- Heun

20

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.4 Classic Runge-Kutta Metho 12.2.4 Classic Runge-Kutta Metho

dd

2 차나 3 차보다 더 정확한 해를 구해 줌 다른 Runge-Kutta 방법들 중에서 가장 널리 사용됨

43211

34

23

12

1

6

1

3

1

3

1

6

1

,2

1,

2

12

1,

2

1

kkkkyy

)k yh hf(x k

)k yh hf(x k

)k yh hf(x k

), y hf(x k

ii

ii

ii

ii

ii

21


dd

MATLAB Function for Classic Runge-Kutta MethodFunction [x, y] = RK4 (f, tspan, y0, n)% solve y’ = f(x, y) with initial condition y(a) = 0% using n steps of the classic 4th order Runge-Kutta method;a = trans(1); b = trans(2); h = (b - a) / nx = (a+h: h : b);k1 = h * feval (f, a, y0);k2 = h * feval (f, a + h / 2, y0 + k1 / 2);k3 = h * feval (f, a + h / 2, y0 + k2 / 2);k4 = h * feval (f, a + h, y0 + k3);y(1) = y0 + k1/6 + k2/3 + k3/3 + k4/6;for i = 1 : n-1 k1 = h * feval (f, x(i), y(i) ); k2 = h * feval (f, x(i) + h/2, y(i) + k1 / 2); k3 = h * feval (f, x(i) + h/2, y(i) + k2 / 2); k4 = h * feval (f, x(i) + h, y(i) + k3); y(i+1) = y(i) + k1/6 + k2/3 + k3/3 + k4/6;endx = [ a x ];y = [ y0 y];

22


dd

Example 12.6 Solving a Simple ODE with the Classic Runge-Kutta Method

0.2h 5,n 1,b 0,a ,2)0( ,),(' yyxyxfy

x Exact solutionApproximate

SolutionAbsolute error

0.2 2.4642 2.4642 8.2745e-06

0.4 3.0755 3.0755 2.0213e-05

0.6 3.8664 3.8663 3.7032e-05

0.8 4.8766 4.8766 6.0308e-05

1.0 6.1548 6.1548 9.2076e-05

<P406 참조 > Exact : 6.1548 Euler’s : 5.7812 Midpoint : 5.922

23

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.5 Other Runge-Kutta Method 12.2.5 Other Runge-Kutta Method

ss

General Runge-Kutta Mothod (1/2)

mmnn

mmmmmnmnm

nn

nn

nn

nn

kwkwkwyy

kakakayhcxhfk

)kakaka yhc hf(x k

)kaka yhc hf(x k

)ka yhc hf(x k

), y hf(x k

22111

11,2211

34324214144

23213133

12122

1

),(

,

,

, c3 a31

w1 w2

c2 a21

a32

w3

c4 a41 a42 a43

w4

24

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD 12.2.5 Other Runge-Kutta Method 12.2.5 Other Runge-Kutta Method

ss

General Runge-Kutta Mothod (2/2)

- Classic fourth-order Runge-Kutta

- Lawson’s fifth-order Runge-Kutta Method

- Kutta’s method

- Butcher’s sixth-order Runge-Kutta method

1/2 0

1/6 2/6

1/2 1/21/2

2/6

1 0 0 1

1/6

1/2 0

1/6 2/6

1/2 1/21/2

2/6

1 0 0 1

1/6

1/4 3/16

7/90 0

1/2 1/2

1/16

32/90

1/2 0 0 1/2

12/90

3/4 0 -3/16 6/16

1 1/7 4/7 6/7

9/16

-12/7 8/7

32/90 7/90

2/3 0

11/120 0

1/3 1/32/3

27/40

1/3 1/12 1/3 -1/12

27/40

1/2 -1/16 9/8 -3/16

1/2 0 9/8 -3/8

-3/8

-3/4 1/2

-4/15 -4/15

1 9/44 -9/11 63/44 18/11 0 -16/11

11/120

25

12.3 Multistep methods12.3 Multistep methods Multistep methods

– 앞에서의 방법론들은 단일점 x = xi 에서의 정보를 이용하여 , x = xi+

1 의 해 yi+1 의 값을 계산했다 . – one step methods

– 단계별로 계산량이 적기 때문에 다른 방법에 비해 효율적

– General form of two-step methods

• a1,a2,b0,b1 등은 methods 에 따라 달라진다 .

• 앞으로는 다음과 같은 notation 으로 표현하겠다 .

)],(),(),([ 11211101211 iiiiiiiii yxfbyxfbyxfbhyayay

n

abhyxf iii

),,(

26

12.3 Multistep methods12.3 Multistep methods Multi-step methods (cont’)

– Multi-step methods 는 fi+1 의 계수 b 의 값에 따라• b = 0 인 경우 명시적 (explicit) 또는 개식 (open) methods 라 부르고 ,

• b ≠0 인 경우에는 암묵적 (implicit) 또는 폐식 (closed) method 라 부른다 .

– Multi-step methods 는 시작 값을 필요로 하므로 , 보통 Runge-Kutta 와 같은 기존의 method 를 이용해서 초기값 y1,y2,… 을 구한다 .

27

12.3 Multistep methods12.3 Multistep methods12.3.1 Adams-Bashforth Methods12.3.1 Adams-Bashforth Methods

Most popular explicit multistep methods Two-step method formula

– Local truncation error O(h3)

Three-step method formula– y0 는 주어져 있고 , y1&y2 는 one-step method 에서 얻는다 .

– Local truncation error O(h4)

Step 이 올라가면 그만큼 truncation error 가 줄어든다 .

.1,....,1],3[2 11 niforffh

yy iiii

.1,....,2],51623[12 211 niforfffh

yy iiiii

28


Mathlab code

29


Example 12.8– Y’ = x+y, y(0) = 2

x 0 0.2 0.4 0.6 0.8 1

y 2 2.46 3.0652 3.8509 4.8546 6.1241

(real)y 2 2.4642 3.0755 3.8664 4.8766 6.1548

30

12.3 Multistep methods12.3 Multistep methods12.3.2 Adams-Moultion Methods12.3.2 Adams-Moultion Methods

Most popular implicit method

Two-step method formula

– y0 is given, y1 is found from a one step methods

– Local truncation error : O(h4) • Adams-Bashforth Methods 의 local truncation error : O(h3)• Adams-Bashforth Methods 에 비해서 error 가 작다

.1,....,1],85[12 111 niforfffh

yy iiiii

31

12.3 Multistep methods12.3 Multistep methods12.3.2 Adams-Moultion Methods12.3.2 Adams-Moultion Methods

Matlab code

32

12.3 Multistep methods12.3 Multistep methods12.3.2 Predictor-Corrector Methods12.3.2 Predictor-Corrector Methods

경우에 따라서 , 앞의 두 다단계 공식을 한 쌍의 공식으로 조합해서 사용하는 것을 Predictor-Corrector Methods 라 한다 .

Procedure

– 각 구간에서 predictor step 으로 implicit method 를 사용한다– Corrector step 으로 explicit method 를 사용한다 . (predictor 가 새 점에서 문제를

예측하고 corrector 가 그 정확도를 개선하는 것을 의미한다 .)

Adams Third-Order Predictor-Corrector Method

– y0 는 초기값으로 주어지고 , y1, y2 는 one-step method 로 구한다 .

– Predictor

– Corrector

.1,....,2],51623[12 211

niforfff

hyy iiiii

]8),(5[12 1111

iiiiii ffyxf

hyy

33

12.4 Stability12.4 Stability Definition

– Root condition :

• M-step method

에 대한 characteristic polynomial

의 근을 λ1, λ2,… λm 로 나타낼때 , | λi| ≤ 1, i = 1,2,…,m 이고 , 절대값이 1 인 모든

근이 simple root 이면 , root condition 을 만족한다 라고 한다 .

– Strongly stable : root condition 을 만족하고 크기가 1 인 특성방정식의 유일한 근으로 λ = 1 인 방법

– Weakly stable : root condition 을 만족하고 크기가 1 인 근을 두개 이상 갖는 방법

– Unstable : root condition 을 만족하지 않는 방법

)...()( 22

11 m

mmm aaaP

]...[... 111011211 mimiimimiii fbfbfbhyayayay

34

12.4 Stability12.4 Stability Example 12.11 – Weakly Stable method

– Simple two step method yi+1 = yi-1 +2hfi

– Differential equation y’ = -4y, y(0) = 1 : exact solution is y = e-4x

– P(λ) = λ2 – (a1λ1 + a2λ0 ) = λ2 – 1 = 0 : λ = 1, -1

ㅡ > weakly stable

35

12.4 Stability12.4 Stability Example 12.11(cont’)

36

12.5 MATLAB 12.5 MATLAB ’’s Methods Method MATLAB includes three functions for solving non-stiff ODEs

– ode 23, ode45• Implement a pair of explicit Runge-Kutta methods

• second and third order and forth and fifth order

– ode113• Fully variable step-size ODE solver based on the Adams-Bashforth-Moulton fa

mily of formulas of orders 1-12

– syntax for function

[t, y] = ode23(‘F’, tspan, y0), where tspan=[t0 t_final]

• ‘F’: string contatining the name of an ODE file

• F(t, y): must return a column vector of values

• each row of solution array y corresponds to a time returned in column vector t

• y0: initial conditions are given in the vector y0

Appendix AAppendix A

38

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD Runge-Kutta 방법의 유도

),(

),(

12

1

kyhxhfk

yxhfk

nn

nn

이라 할 때 ,

211 kkyy nn

여기서 실수 a, b, α, β 는 (1) 를 Taylor 급수 전개하였을 때 ,가능한 한 Taylor 방법과 같아 지도록 결정되어져야 할 상수이다 .이제 , y(xn+1) 을 투 근방에서 Taylor 급수 전개하면 ,

)()2(6

)(2

)(

)('''6

)(''2

)(')(

4223

2

32

1

hOfffffffffh

fffh

hfxy

xyh

xyh

xhyxyy

yyxyyxyxx

yxn

nnnnn

이다 .

(1)

(2)

39

12.2 RUNGE-KUTTA METHOD12.2 RUNGE-KUTTA METHOD

한편 , 2 변수 함수의 Taylor 전개를 이용하면 ,

)(22

22),(

),(

32

12

11

12

hOfk

fkhfh

fkhfyxf

kyhxfh

k

yyxyxxyxnn

nn

이므로 , k2 를 (1) 에 대입하면

)()22

(

)()(1

4222

3

2

hOfffffbh

fffbhhfbaynyn

yyxyxx

yx

을 얻는다 . 이제 , (1) 식과 (2) 식을 비교하여 h 항과 h2 항을 같도록 하려면

1ba

2

1 bb

(3),있으나 여러가지가 값은 β의 α와 b, a, 상수 만족하는 (4)식을

. MethodMidpoint

, 1 ,2

1

된다가

택하면로 ba

40

Truncation errorTruncation error 수치해 ui 에 포함된 오차 ei

전체 절단오차– 단계의 수가 (i+1) 이라면 , Xi+1 에서 전체 절단오차는 각 단계 발생오차와 계산 단계 수를 곱한

것이라 할 수 있다 .

– 로 부터 를 얻을 수 있으므로 각 단계

발생오차와 계산 단계 수를 곱하면 전체 절단 오차는 이다 . →first order method

),(''2

)},(),({)(

),()},(''2

),({

),(

11

2

111111

11111

2

111

111

iiiiiiii

iiiiiiii

iiii

ii

i

yxyh

uxfyxfhuy

uxhfuyxyh

yxhfy

uxhfuy

uy

e)( 2hO

hixxi

)1(01

h

xxi i

)()1( 01

)(hO

Documents

12 Ordinary Differential Equations: Initial-Value Problems 2005. 06. 01 자연어처리 연구실 김혜겸 윤도상 최성원