lllÑÑÑmmm Markov óóó

ÅÅÅLLL§§§µµµ111888ÙÙÙ

���ááá���

nnnÆÆÆ���êêêÆÆÆXXXVVVÇÇÇÚÚÚOOO���ïïï¿¿¿

ëëëÖÖÖµµµ555ÅÅÅLLL§§§-OOOAAA^̂̂666

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

SN

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

lÑm Markov ó X = {Xn : n ∈ N∗} ½Â:

(a). m¢Ú8Ü: T = N∗ = {0, 1, 2, . . . }

(b). G�m: lÑê8Ü S = {a0, a1, . . . }

(c). ∀ n ∈ N∗ 9 i0, . . . , in, in+1 ∈ S ,

P(Xn+1 = in+1|X0 = i0,X1 = i1, · · · ,Xn = in)= P(Xn+1 = in+1|Xn = in)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

k Ú=£VÇ:

p(k)ij (n) = P(Xn+k = j |Xn = i), i , j ∈ S

1 Ú=£VÇ: pij(n) := p(1)ij (n)

0 Ú=£VÇ:

p(0)ij (n) = P(Xn = j |Xn = i) = δij =

{1, i = j ,0, i 6= j , i , j ∈ S

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

k Ú=£VÇÝ:

P(k)(n) =[p(k)ij (n)

]i ,j∈S

k Ú=£VÇÝ´ÅÝ (random matrix):

p(k)ij (n) ≥ 0,

∑j∈S

p(k)ij (n) = 1

0 Ú=£VÇÝ´ü :

P(0)(n) = I

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

àg Markov ó:

Markov ó X � k Ú=£VÇ p(k)ij (n) Ø6 n:

p(k)ij = P(Xn+k = j |Xn = i)

= P(Xm+k = j |Xm = i), ∀ m, n ≥ 0= P(Xk = j |X0 = i)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

~: Õá Markov ó

� X0, . . . ,Xn, . . . ´ i.i.d. Щ©Ù

P(X0 = i) = ai , i = 1, 2, . . . ,m

K X = {Xn : n ∈ N∗} ´àgê¼ó.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

y:

(a). àgMarkov 5: ∀ i0, . . . , in ∈ S = {1, . . . ,m},

P(Xn = in|X0 = i0, · · · ,Xn−1 = in−1) = P(Xn = in) = ain= P(Xn = in|Xn−1 = in−1) = P(Xn+k = in|Xn+k−1 = in−1)= pin−1in .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(b). 1 Ú=£VÇÝ:

P =

a1 a2 · · · am−1 ama1 a2 · · · am−1 am· · · · · · · · · · · · · · ·· · · · · · · · · · · · · · ·a1 a2 · · · am−1 am

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

~: ÅiÄ

� r.v.s (ξn; n ≥ 1) i.i.d. u

P(ξ1 = k) = ak , k = Z.

ÅiĽÂ

S0 = 0, Sn =n∑

i=1

ξi , n ≥ 1

K S = {Sn : n ≥ 0} ´àgê¼ó.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

y:

(a). ÅiÄ5: ξn (Sm : m ≤ n − 1) Õá

(b). ÅiÄ5: Sn = Sn−1 + ξn, ∀ n ≥ 1

(c). Markov 5: ∀ i0, . . . , in ∈ S = Z,

P(Sn = in|S0 = 0,S1 = i1, · · · ,Sn−1 = in−1)= P(ξn + in−1 = in|S0 = 0,S1 = i1, · · · ,Sn−1 = in−1)= P(ξn = in − in−1|S0 = 0,S1 = i1, · · · ,Sn−1 = in−1)= P(ξn = in − in−1) = ain−in−1 .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(d). àg5: i , j ∈ S = Z,

P(Sn = j |Sn−1 = i) = P(ξn + in−1 = j |Sn−1 = i)= P(ξn = j − i |Sn−1 = i)= P(ξn = j − i) = aj−i independent of n

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(e). 1 Ú=£VÇÝ:

P =

· · · · · · · · · · · · · · · · · ·· · · a0 a1 · · · an−1 · · ·· · · a−1 a0 · · · an−2 · · ·· · · · · · · · · · · · · · · · · ·· · · · · · · · · · · · · · · · · ·· · · · · · · · · · · · · · · · · ·· · · a1−n a2−n · · · a0 · · ·· · · · · · · · · · · · · · · · · ·

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

~: lÑüè�. M/G/1

ykÑÖ�. ��ÑÖ�güè�ÉÑÖ, ÑÖ�Uì k�kÑÖ (FIFO) �K. ^ Xt L« t XÚ�ê ()�3�ÉÑÖ��). ��ÉÑÖ�lmmP

T0 < T1 < T2 < · · · < Tn < · · · .

^ An+1 L« Tn+1 lm���ÉÑÖÏm�XÚ��ê.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

P Xn = XTn+. K Xn L« Tn XÚo�ê ()Tn=òlm�@ �). u´ Xn ÷v

Xn+1 = (Xn − 1)+ + An+1.

b� (An : n ≥ 1) i.i.d. Õáu X0, Ù©ÙÇ

P(A1 = k) = ak , k ≥ 0,

K X = {Xn : n ≥ 0} ´àgê¼ó.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

y:

(a). �.{z: r Xn+1 �¤

Xn+1 = g(Xn,An+1), g(x , y) = (x − 1)+ + y .

ù�

X1 = g(X0,A1), X2 = g(X1,A2) = g(g(X0,A1),A2), · · · ,

ù`²: Xn ´ X0, A1, . . . ,An �¼ê. �

X0,X1,X2, . . . ,Xn ÑÕáu An+1.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(b). Markov 5: ∀ i0, . . . , in ∈ N∗,

P(Xn+1 = in+1|X0 = i0,X1 = i1, · · · ,Xn = in)= P(g(Xn,An+1) = in+1|X0 = i0,X1 = i1, · · · ,Xn = in)= P(g(in,An+1) = in+1|X0 = i0,X1 = i1, · · · ,Xn = in)= P(g(in,An+1) = in+1|Xn = in) = P(Xn+1 = in+1|Xn = in)= P(g(in,An+1) = in+1)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(c). àg 5: ∀ j ≥ i ∈ N∗,

pij = P(Xn+1 = j |Xn = i) = P(g(i ,An+1) = j |Xn = i)= P(g(i ,An+1) = j) = P(g(i ,A1) = j)= P(g(i ,A1) = j |X0 = in) = P(g(X0,A1) = j |X0 = i)= P(X1 = j |X0 = i) = aj−(i−1)+

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov ó½Â

(e). 1 Ú=£VÇÝ:

P =

a0 a1 a2 · · · · · ·a0 a1 a2 · · · · · ·0 a0 a1 · · · · · ·0 0 a0 a1 · · ·· · · · · · · · · · · · · · ·

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

Ìù–�#xâŧ (C -K §):

p(k+m)ij (n) =

∑l∈S

p(k)il (n)p

(m)lj (n + k), i , j ∈ S

Ý/ª:

P(k+m)(n) = P(k)(n) · P(m)(n + k)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

y:

∀ n, k ,m ≥ 0 9 i , j ∈ S ,

p(k+m)ij (n) = P(Xn+k+m = j |Xn = i)= P {∪l∈S(Xn+k = l),Xn+k+m = j |Xn = i}= P {∪l∈S(Xn+k = l ,Xn+k+m = j)|Xn = i}=∑l∈S

P(Xn+k = l |Xn = i)P(Xn+k+m = j |Xn = i ,Xn+k = l)

=∑l∈S

p(k)il (n)p

(m)lj (n + k)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

àg Markov ó C -K §:

p(k+m)ij =

∑l∈S

p(k)il p

(m)lj , i , j ∈ S

Ý/ª:P(k+m) = P(k) · P(m)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

k Ú=£VÇd 1 Ú=£VÇ(½:

P(k) = Pk , ∀ k ≥ 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

Щ©Ù:q(0)j = P(X0 = j), j ∈ S

Щ©Ùþ:

q(0) =[q(0)1 , q

(0)2 , · · · , q

(0)j , · · ·

]

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

Theorem

Markov ó X �k©Ùd§�Щ©ÙÚ 1 Ú=£VÇ��(½:

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

y: ∀ n1 < n2 < · · · < nm 9 i1, · · · , im, i ∈ S ,

P(Xn1 = i1,Xn2 = i2, · · · ,Xnm = im)= P(∪i∈S(X0 = i),Xn1 = i1,Xn2 = i2, · · · ,Xnm = im)=∑i∈S

P(X0 = i ,Xn1 = i1,Xn2 = i2, · · · ,Xnm = im)

=∑i∈S

q(0)i p

(n1)ii1

(0)p(n2−n1)i1i2

(n1) · · · p(nm−nm−1)im−1im (nm−1)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

ýé©Ù:q(n)j = P(Xn = j), j ∈ S

ýé©Ùþ:

q(n) =[q(n)1 , q

(n)2 , · · · , q

(n)j , · · ·

]

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

Markov ó X ýé©ÙdЩ©ÙÚ 1 Ú=£VÇ(½:

q(n) = q(0) · P(n) = q(0) · Pn

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

y:

∀ n ≥ 0,

q(n)j = P(Xn = j) = P {∪i∈S(X0 = i),Xn = j}

= P {∪i∈S(X0 = i ,Xn = j)} =∑i∈S

P(X0 = i ,Xn = j)

=∑i∈S

q(0)i p

(n)ij (0),

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

~K:

�àg Markov ó X G�m S = {0, 1, 2},

P =

23 13 013

13

13

0 1212

,e X Щ©Ù q0 = q1 = q2 =

13 , O P(X2 = 3).

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

):

(a). 2 Ú=£VÇÝ:

P2 = PP =

59 13 019

718

518

16

512

512

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óVÇ©Ù

(b). ^^VÇ:

P(X2 = 2) =2∑

i=0

P(X0 = i)P(X2 = 2|X0 = i)

= q(0)0 · p

(2)02 + q

(0)1 · p

(2)12 + q

(0)2 · p

(2)22

=1

3×(1

9+

5

18+

5

12

)=

29

108

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

±�

(a). G�mL«: S = {1, 2, · · · }

(b). Markov ó´àg�

Äm (Markov time):

τj = min{m ≥ 1; Xm = j}, j ∈ S

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

Ä5:

(a). ∀ n ≥ 1 9 j ∈ S ,

{τj = n} = {Xm 6= j ; m = 1, . . . , n − 1} ∩ {Xn = j}

(b). ∀ n ≥ 1 9 j ∈ S ,

{Xn = j} ⊂ {τj ≤ n}

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

ÄVÇ: ∀ i , j ∈ S Ú n ≥ 1,

f(n)ij = P(τj = n|X0 = i)

= P(Xn = j , Xm 6= j ; m = 1, 2, . . . , n − 1|X0 = i)

´@VÇ: ∀ i , j ∈ S ,

fij =∞∑n=1

f(n)ij = P(τj

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

ØVÇ: ∀ i , j ∈ S ,

f(∞)ij = P(τj =∞|X0 = i)

= P(Xn 6= j ; ∀ n ≥ 1|X0 = i)

²þ=£Úê: ∀ i , j ∈ S ,

µij = E [τj |X0 = i ] =∞∑n=1

nf(n)ij

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~G� (recurrent state):

i ∈ S ´~G�, XJ fii = 1

~G�)º:

P(Xn = i for infinitely many n|X0 = i) = 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~G� (transient state):

i ∈ S ´~G�, XJ fii < 1

~G�)º:

P(Xn = i for infinitely many n|X0 = i) = 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

�~G� (positive recurrent state):

(a). i ∈ S ´~G� (fii = 1)

(b). ²þ=£Úê µii < +∞

"~G� (null recurrent state):

(a). i ∈ S ´~G� (fii = 1)

(b). ²þ=£Úê µii = +∞

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~K:

Markov ó X G�m S = {1, 2, 3, 4},

P =

12

12 0 0

1 0 0 00 13

23 0

12 0

12 0

,©Û~Ú~G�.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

):

(a). â P xG�=£ã

(b). lG�=£ãOG� 1 ÄÚ´@VÇ:

f(1)11 = p11 =

1

2, f

(2)11 =

1

2, f

(n)11 = 0 (n ≥ 3),

f11 =∞∑n=1

f(n)11 = 1, µ11 =

∞∑n=1

nf(n)11 = 1

�G� 1 ´�~�.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(c). G� 2 ÄÚ´@VÇ:

f(1)22 = 0, f

(2)22 =

1

2, f

(n)22 =

1

2n−1(n ≥ 3),

f11 =∞∑n=1

f(n)11 = 1, µ11 = 1 +

3

22+ · · ·+ n

2n−1+ · · · = 3

�G� 2 ´�~�.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(d). G� 3 ÄÚ´@VÇ:

f(1)33 =

2

3, f

(n)33 = 0, (n ≥ 2)

f33 =2

3< 1

�G� 3 ´~�.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(e). G� 4 ÄÚ´@VÇ:

f(n)44 = 0, (n ≥ 1)

f44 = 0 < 1

�G� 4 ´~�.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

G��±Ï:

G� i ∈ S �±Ï½Â:

di = GCD{n|n ≥ 1; p(n)ii > 0}

(a). G� i ∈ S äk±Ï: di > 1

(b). G� i ∈ S ±Ï: di = 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

±Ï�5:

(a). e p(n)ii > 0, K3 m ∈ N ¦ n = mdi

(b). 3 N0 ≥ 1 ¦ p(ndi )ii > 0, ∀ n ≥ N0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b)�y²:

(b1). ò {n|n ≥ 1; p(n)ii > 0} ¥�êl��ü�:

{n1, n2, n3, . . . , nk , . . . }.

(b2). ½Â tk = GCD{n1, n2, . . . , nk}, ∀ k ≥ 1, K

t1 ≥ t2 ≥ · · · ≥ di ≥ 1.

(b3). 3 N ≥ ¦ tN = tN+1 = · · · = d �

di = GCD{n1, n2, . . . , nN}.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b4) âÐ�êØ: é� N0 ≥ 1 ¦ ∀ n ≥ N0,

nd =N∑

k=1

aknk , ak ∈ N

(b5) d C -K §:

p(nd)ii = p

(∑N

k=1 aknk )ii ≥

N∏k=1

p(aknk )ii ≥

N∏k=1

(p(nk )ii

)ak> 0.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(c). e3 n ≥ 1 ¦p(n)jj > 0 Ú p(n+1)jj > 0, K dj = 1

(d). e3 m ≥ 1 ¦ P(m) ¥1 j ���Ø", K dj = 1.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(d) �y²:

(d1). dK¿: ∀ i ∈ S , k p(m)ij > 0, K3 i1 ∈ S ¦ pii1 > 0

(d2). d C -K §9 p(m)i1j

> 0,

p(m+1)ij =

∞∑l=1

pilp(m)lj ≥ pii1p

(m)i1j

> 0

þª� i = j k p(m+1)jj > 0 9 p

(m)jj > 0, �d (c), dj = 1.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

Markov óG�©a:

S 3 i

~ (fii < 1)

~ (fii = 1)

"~ (µii =∞)�~ (µii 1)H{ (di = 1)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

=£VÇÄVÇ'X:

(a). ∀ i , j ∈ S 9 n ≥ 1,

f(n)ij ≤ p

(n)ij ≤ fij

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

f(n)ij = P(τj = n|X0 = i)

= P(Xn = j , Xm 6= j , m = 1, 2, . . . , n − 1|X0 = i)≤ P(Xn = j |X0 = i) = p(n)ij≤ P(τj ≤ n|X0 = i)≤ P(τj

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b). ∀ i , j ∈ S 9 n ≥ 1,

f(n)ij =

∑i1 6=j

∑i2 6=j

· · ·∑

in−1 6=j

pii1pi1i2 · · · pin−1j

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(b1) ky²� n > 1,

f(n)ij =

∑i1 6=j

pii1 f(n−1)i1j

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

f(n)ij = P(X1 6= j , X2 6= j , · · · , Xn−1 6= j , Xn = j |X0 = i)

=∑i1 6=j

P(X1 = i1, X2 6= j , · · · , Xn−1 6= j , Xn = j |X0 = i)

=∑i1 6=j

P(X2 6= j , · · · , Xn−1 6= j , Xn = j |X0 = i ,X1 = i1)

×P(X1 = i1|X0 = i) Markov property=

∑i1 6=j

P(X2 6= j , · · · , Xn−1 6= j , Xn = j |X1 = i1)pii1

=∑i1 6=j

pii1P(X1 6= j , · · · , Xn−2 6= j , Xn−1 = j |X0 = i1)

=∑i1 6=j

pii1 f(n−1)i1j

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b2) é (b1) S:

f(n)ij =

∑i1 6=j

pii1 f(n−1)i1j

=∑i1 6=j

pii1

∑i2 6=j

pi1i2 f(n−2)i2j

=

∑i1 6=j

∑i2 6=j

pii1pi1i2 f(n−2)i2j

= · · ·

=∑i1 6=j

∑i2 6=j

· · ·∑

in−1 6=j

pii1pi1i2 · · · pin−2in−1 f(1)in−1j

=∑i1 6=j

∑i2 6=j

· · ·∑

in−1 6=j

pii1pi1i2 · · · pin−2in−1pin−1j .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(c). ∀ i , j ∈ S 9 n ≥ 1,

p(n)ij =

n∑l=1

f(l)ij p

(n−l)jj

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(c1) 5¿�'X:

{Xn = j} ⊂ ∪nl=1{Xm 6= j , m = 1, . . . , l − 1, Xl = j}

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(c2) ^ (c1) :

p(n)ij = P(∪

nl=1(Xm 6= j , m = 1, . . . , l − 1, Xl = j),Xn = j |X0 = i)

=n∑

l=1

P(Xm 6= j , m = 1, . . . , l − 1, Xl = j ,Xn = j |X0 = i)

=n∑

l=1

P(Xn = j |Xm 6= j , m = 1, . . . , l − 1, Xl = j ,X0 = i)

×P(Xm 6= j , m = 1, . . . , l − 1, Xl = j |X0 = i)

=n∑

l=1

P(Xn = j |Xl = j)f (l)ij =n∑

l=1

f(l)ij p

(n−l)jj .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~G��O:

(a). ∀ i ∈ S ,∞∑n=1

p(n)ii =

1

1− fii

(b). d (a):

i ∈ S ~ ⇐==⇒∞∑n=1

p(n)ii = +∞,

i ∈ S ~ ⇐==⇒∞∑n=1

p(n)ii < +∞

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(a) �y²:

(a1) � p(0)ij = δij Ú f

(0)ij = 0, K½ÂÝ1¼ê:

Pij(x) =∞∑k=0

p(k)ij x

k , Fij(x) =∞∑k=0

f(k)ij x

k , 0 < x < 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(a2)

Pij(x) =∞∑n=0

p(n)ij x

n = p(0)ij +

∞∑n=1

p(n)ij x

n

= δij +∞∑n=1

(n∑

m=1

f(m)ij p

(n−m)jj

)xmxn−m

= δij +∞∑

m=0

f(m)ij x

m∞∑n=0

p(n)jj x

n, f(0)ij = 0

= δij + Fij(x)Pjj(x), 0 < x < 1.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(a3) - x ↑ 1 9 limx→1 Fij(x) = fij ,

∞∑n=0

p(n)ij = δij + fij

∞∑n=0

p(n)jj

⇐==⇒ δij +∞∑n=1

p(n)ij = δij + fij

(1 +

∞∑n=1

p(n)jj

)

⇐==⇒∞∑n=1

p(n)ij = fij

(1 +

∞∑n=1

p(n)jj

).

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~G�±Ï�Úê=£VÇ4:

� i ∈ S ´~G�, K

limn→∞

p(ndi )ii =

diµii

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

�!"~9H{G��O: � i ∈ S ~G�

(a). i ´"~� ⇐==⇒ limn→∞ p(n)ii = 0

(b). i ´H{� ⇐==⇒ limn→∞ p(n)ii =1µii

(c). i ´�~±Ï� ⇐==⇒ limn→∞ p(n)ii Ø3

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(a) �y²:

(a1) � i "~, K limn→∞ p(ndi )ii =

di+∞ = 0. e m Ø´ di �ê,

Kd±Ï½Â: p(m)ii = 0. nÜ: limn→∞ p

(n)ii = 0

(a2) � limn→∞ p(n)ii = 0, b� i Ø´"~, K i �"~.

� limn→∞ p(ndi )ii =

diµii> 0, ù limn→∞ p

(n)ii = 0 gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b) �y²:

(b1) � i H{, K limn→∞ p(ndi )ii = limn→∞ p

(n)ii =

1µii> 0

(b2) � limn→∞ p(n)ii =

1µii> 0, d (a) : i Ø´"~�, �Ù

�~. ,d4�Ò5: 3¿© n ¦

p(n)ii > 0, p

(n+1)ii > 0

Ïd di = 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(c) �y²:

(c1) � i �~±Ï�, b� limn→∞ p(n)ii 3, K

limn→∞

p(n)ii ≥ 0

e limn→∞ p(n)ii = 0, d (a) : i ´"~, ��)gñ.

e limn→∞ p(n)ii > 0, d4��Ò5: di = 1, �)gñ

(c2) � limn→∞ p(n)ii Ø3, d (a) : i Ø´"~�; d (b) :

i Ø´H{�, �ÙU´�~±Ï�

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

!"~G�=£VÇ4:

e j ∈ S ´"~½~, K

limn→∞

p(n)ij = 0, ∀ i ∈ S

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(a) j = i :

e j ~ ⇐==⇒∑∞

n=1 p(n)jj

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b) j 6= i : ∀ 1 < n′ < n,

p(n)ij =

n∑l=1

f(l)ij p

(n−l)jj ≤

n′∑l=1

f(l)ij p

(n−l)jj +

n∑l=n′+1

f(l)ij

�½ n′, - n→∞ �þ4,

lim supn→∞

p(n)ij ≤

n′∑l=1

f(l)ij · 0 +

∞∑l=n′+1

f(l)ij ,

2- n′ →∞, lim supn→∞ p(n)ij = 0, qÏ p

(n)ij ≥ 0,

� lim infn→∞ p(n)ij = 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

±ÏL«

e��ê8Ü{

n|n ≥ 1; f (n)ii > 0}, P

hi = GCD{

n|n ≥ 1; f (n)ii > 0}

K di Ú hi ¥e3, ,3, di = hi .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

G�: ∀ i , j ∈ S ,

e n ∈ N ¦ p(n)ij > 0, K i j , P i → j

G�pÏ: ∀ i , j ∈ S ,

e i → j 9 j → i , K i j pÏ, P i ↔ j

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

pÏ5:

(a). �D45:

i → j , and j → k ==⇒ i → k

(b). p�D45:

i ↔ j , and j ↔ k ==⇒ i ↔ k

(c). pÏ�é¡5:i ↔ j ==⇒ j ↔ i

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

^´@VÇ�O:

(a). ∀ i , j ∈ S ,i → j ⇐==⇒ fij > 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(a1) b� i → j , K3 n ≥ 1 ¦ p(n)ij > 0, u´:

fij ≥ p(n)ij > 0.

(a2) b� fij > 0, Kd fij =∑∞

n=1 f(n)ij > 0 �:∃ n ≥ 1 ¦ f

(n)ij > 0,

�p(n)ij ≥ f

(n)ij > 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b). e i 6= j j ~±9 j → i , K

i ↔ j , and fij = fji = 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(b1) ky² i → j : ^y{, b� i 9 j , ù¿X

P(Xl 6= j ; ∀ l ≥ 1|X0 = i) = 1

(b2) du j → i , K3 m ≥ 1 ¦ p(m)ji > 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b3) 5¿ j ~, K

0 = P(Xl 6= j ; ∀ l ≥ 1|X0 = j)= P(Xl 6= j ; ∀ l ≥ m + 1|X0 = j), áõg� j≥ P(Xl 6= j ; ∀ l ≥ m + 1,Xm = i |X0 = j), i 6= j= P(Xl 6= j ; ∀ l ≥ m + 1|Xm = i ,X0 = j)P(Xm = i |X0 = j)= P(Xl 6= j ; ∀ l ≥ m + 1|Xm = i)p(m)ji Markov property

= P(Xl 6= j ; ∀ l ≥ 1|X0 = i)p(m)ji= p

(m)ji > 0. �)gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b4) �Xy² fij = 1: du j → i , Kd (a), 3 m ≥ 1 ¦

f(m)ji = P(τi = m|X0 = j) > 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(b5) d j ~, K

0 = 1− fjj = P(Xn 6= j ; ∀ n ≥ 1|X0 = j)= P(Xn+m 6= j ; ∀ n ≥ 1|X0 = j)≥ P(τi = m,Xn+m 6= j ; ∀ n ≥ 1|X0 = j)= P(Xn+m 6= j ; ∀ n ≥ 1|τi = m,X0 = j)P(τi = m|X0 = j)= P(Xn+m 6= j ; ∀ n ≥ 1|Xm = i)f (m)ji= P(Xn 6= j , ∀ n ≥ 1|X0 = i)f (m)ji= (1− fij)f (m)ji

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

pÏG�äkÓG�a.:

� i , j ∈ S, i ↔ j , K i Ú j ½öÓ~, ½öÓ"~,½öÓ�~�±Ï�±ÏÓ, ½öÓH{.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

y:

(a) i ↔ j , K3 l , n ≥ 1 ¦ α = p(l)ij > 0 Ú β = p(n)ji > 0

(b) d C -K §:

p(l+m+n)ii =

∑k

∑s

p(l)ik p

(m)ks p

(n)si ≥ p

(l)ij p

(m)jj p

(n)ji = αβp

(m)jj

(c) q/: i j  

p(l+m+n)jj ≥ αβp

(m)ii

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(d) e j ~, K∑∞

m=1 p(m)jj = +∞. d (b):

∞∑m=1

p(l+m+n)jj = +∞, d= i ~

(e) e j "~, K limm→∞ p(m)jj = 0 Ú limm→∞ p

(l+m+n)jj = 0,

^ (c) �: limm→∞

p(m)ii = 0, � i "~

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

(f) e i , j �~, K di = dj . ¯¢þ, ^ C -K §:

p(l+n)jj =

∑k

p(n)jk p

(l)kj ≥ p

(n)ji p

(l)ij = αβ > 0

� dj �Ø n + l . � m ∈ {n|n ≥ 1; p(n)ii > 0}, Kd (c) �:

p(l+m+n)jj ≥ αβp

(m)ii > 0

¤± dj �Ø l +m + n, �U�Ø m, =k dj ≤ di . Óny:di ≤ dj .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

~K:

�ê¼ó X �G�m S = {0, 1, 2, · · · }, G�=£VÇ

pii+1 =1

2, pi0 =

1

2, i = 0, 1, 2, · · · .

�ä X G��a..

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�©a

):

(a) xG�=£ã

(b) lG�=£ã: ¤kG�pÏ

(c) �O 0 G�: p00 = 1/2 > 0, � d0 = 1,

f00 =∞∑n=0

f(n)00 =

∞∑n=0

1

2n= 1, µ00 =

∞∑n=1

n1

2n= 2 < +∞

u´¤kG�þH{� (¡ H{ó)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

48�48:

(a). C ⊂ S ´48:

∀ i ∈ C 9 j ∈ C , k p(n)ij = 0, ∀ n ≥ 0

(b). C ⊂ S ´Ø�48:

48 C ¥Ø¹?Û�4ýf8

(c). áÂ�:

e48 C kG�, K¡TG�áÂ�

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

Ø� Markov ó:

G�m S ´Ø�48� Markov ó

48 C ��O:

(a). C ´48 ⇐=⇒ pij = 0, ∀ i ∈ C 9 j ∈ C

(b). C ´48 ⇐=⇒∑

j∈C pij = 1, ∀ i ∈ C

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y (a):

(a1) e C ´48, Kd48½Â: pij = 0, ∀ i ∈ C 9 j ∈ C

(a2) b� pij = 0, ∀ i ∈ C 9 j ∈ C . ^êÆ8B{, � p(k)ij = 0,∀ i ∈ C 9 j ∈ C , Kd C -K §:

p(k+1)ij =

∑l∈S

p(k)il plj =

∑l∈C

p(k)il plj +

∑l∈C

p(k)il plj

=∑l∈C

p(k)il · 0 +

∑l∈C

0·lj = 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

G�m��da©):

(a). pÏ´«�d'X: g5!é¡5!D45

(b). pÏ�da©):

S = ∪nSn, Sm ∩ Sn = ∅, m 6= n

Ù¥ �da Sn ¥G�pÏ (ÏdäkÓG�a.)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

pÏ�da©)�"::

�da Sm Ú Sn (m 6= n) ¥�G�U3ü

�da´Ø�48��O:

¹~G���da Sn ´Ø�48

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y:

(a) ¹~G���da Sn ´48:

¹~G�� Sn ¤kG�Ñ~. ∀ i ∈ Sn 9 ∀ j ∈ S ,e i → j , d i ~, K i ↔ j , � j ∈ S

(b) ¹~G���da Sn ´�48:

� C ⊂ Sn ´?¿48. ?� k ∈ Sn 9 j ∈ C , d Sn ´�da, K j ↔ k . qdu C ´48, � k ∈ C , = Sn ⊂ C . Ïd Sn = C , = Sn عý48.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

G�mØ�48©):

S = D ∪ C1 ∪ C2 ∪ · · ·

Ù¥ D ¤k~��¤�G�f8, Cn (i = 1, 2, . . . ) þ´d ~� �¤�Ø�48

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

kG� Markov óG�m©):

(a). ~��¤�G�f8 D ØU´48

(b). Ø3"~G�

(c). e Markov óØ�, KÙ¤kG�þ´�~

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y (a):

^y{, b� D ´48, K ∀ i ∈ D 9 n ≥ 0,∑j∈D

p(n)ij = 1

∀ j ∈ D, k limn→∞ p(n)ij = 0. 5¿� D �k, éþªü>�4 n→∞ �: 0 = 1, �)gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y (b):

^y{, b� S ¥3"~� j ∈ S , u´3,�da Sn. Ï j ~, � Sn ´48. � ∀ i ∈ Sn 9 m ≥ 0,∑

k∈Sn

p(m)ik = 1

du limm→∞ p(m)ik = 0 9 Sn �êk, �: 0 = 1 gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

~K:

� Markov ó X G�m S = {1, 2, 3, 4}, Ú=£VÇÝ

P =

12

12 0 0

13

23 0 0

14

14

14

14

0 0 0 1

,é X �G�m©).

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

(a). xG�=£ã:

(b). G�m©):

S = D ∪ C1 ∪ C2 = {3} ∪ {1, 2} ∪ {4}

Ù¥ D = {3} ~�f8, C1 = {1, 2} H{�Ø�48, C2 = {4} H{áÂ�

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

Ø�48�±Ï©a:

� C ⊂ S ´±Ï d Ø�48, K C ©) d pØ�G�f8 J1, . . . , Jd ¿, =

C = ∪dm=1Jm, Jm ∩ Jl = ∅, m 6= l

∀ k ∈ Jm (m = 1, 2, . . . , d) k∑j∈Jm+1

pkj = 1, Jd+1 := J1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y:

(a). Jm ��E:

∀ i ∈ C Ú m = 1, · · · , d , ½Â

Jm ={

j ∈ C : ∃ n ≥ 0, p(nd+m)ij > 0},

= Jm ´l i Ñu, 31 m,m + d , · · · ,m + nd , · · · ÚU��G���N

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

(b). Jm 6= Jl , ∀ m 6= l :

b�3 j ∈ Jm ∩ Jl , Kd Jm ½Â: 3 n1, n2 ≥ 0 ¦

p(n1d+m)ij > 0, p

(n2d+m)ij > 0.

u´ d | l −m. �du 1 ≤ m, l ≤ d , � m = l

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

(c).∑

j∈Jm+1 pkj = 1, ∀ k ∈ Jm

d C 48,

1 =∑j∈C

pkj =∑

j∈Jm+1

pkj +∑

j∈C/Jm+1

pkj

d Jm �½Â: 3 n ≥ 0 ¦ p(nd+m)ik > 0, 2d Jm+1 ½Â: � j ∈ C/Jm+1 ,

0 = p(nd+m+1)ij ≥ p

(nd+m)ik pkj ≥ 0

= j ∈ C/Jm+1 pkj = 0.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

~K:

� Markov ó X G�m S = {1, 2, · · · , 6}, Ú=£VÇÝ

P =

0 0 1 0 0 00 0 0 0 0 10 0 0 0 1 013

13 0

13 0 0

1 0 0 0 0 00 12 0 0 0

12

,

©) X G�m, ¿ÑG�f8�a..

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

y:

(a). xG�=£ã:

(b). lG�=£ã©)G�m:

S = D ∪ C1 ∪ C2 = {4} ∪ {1, 3, 5} ∪ {2, 6},

Ù¥ C1 = {1, 3, 5} ±Ï 3 ��~��¤�Ø�48,C2 = {2, 6} H{��¤�Ø�48

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

Markov óG�m©)

(c). éØ�48 C1 = {1, 3, 5} ±Ï©):

J1 = {1}, J2 = {3}, J3 = {5}

=£5K:

J1 → J2 → J3 → J1 → J2 → · · ·

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

1 lÑm Markov óMarkov ó½ÂMarkov óVÇ©ÙMarkov óG�©aMarkov óG�m©)4²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

~K: ¦=£VÇ4

� Markov ó X G�m S = {1, 2, 3, 4}, Ú=£VÇÝ

P =

1 0 0 00 1 0 013

23 0 0

14

14 0

12

?Ø4 limn→∞ p

(n)i1 (i = 1, 2, 3, 4) ´Ä3? e3´Ä

ЩG� i k'?

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

):

(a). xG�=£ã:

(b). G�m©):

S = D ∪ C1 ∪ C2 = {3, 4} ∪ {1} ∪ {2}

Ù¥ 1, 2 þáÂ�

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c). d 1 ´áÂ�, �

p(n)11 = 1, p

(n)21 = 0, p

(n)31 =

1

3

Ïd

limn→∞

p(n)11 = 1, limn→∞

p(n)21 = 0, limn→∞

p(n)31 =

1

3

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(d). ¦ limn→∞ p(n)41 :

p(n)41 =

n∑l=1

f(l)41 p

(n−l)11 =

n∑l=1

f(l)41

=1

4+

1

2· 14+

1

2· 12· 14+ · · ·+ 1

2(n−1)· 14

=1

2− 1

2(n+1)

Ïd limn→∞ p(n)41 =

12

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

=£VÇ4:

� j �~, K ∀ i ∈ S , k

limn→∞

p(ndj+r)ij = fij(r)

djµjj, r = 1, 2, . . . , dj

Ù¥ fij(r) =∑∞

n=0 f(ndj+r)ij ÷v

∑djr=1 fij(r) = fij

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y:

(a). Ún:

� (an; n ≥ 0) Ú (bn; n ≥ 0) ´Kê�. XJ§÷v:

(1)∑∞

n=0 an < +∞ (½∑∞

n=0 an = +∞ � an k.);

(2) limn→∞ bn = b < +∞, K

limn→∞

∑nm=0 ambn−m∑n

m=0 am= b

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(b). 5¿�:

p(ndj+r)ij =

n∑m=1

f(mdj+r)ij p

((n−m)dj )jj

�

an = f(ndj+r)ij k., bn = p

(ndj )jj → b =

djµjj, n→∞

@o

p(ndj+r)ij =

n∑m=0

ambn−m

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c). dþ¡�Ún, ¿5¿�:

n∑m=0

am =n∑

m=0

f(mdj+r)ij → fij(r), n→∞

u´

limn→∞

p(ndj+r)ij = limn→∞

[(n∑

m=0

am

) ∑nm=0 ambn−m∑n

m=0 am

]

= bfij(r) = fij(r)djµjj

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

=£VÇ4o(:

limn→∞

p(n)ij =

0, j ∈ D ∪ C0, i ∈ Sfijµjj, j ∈ Cm H{, i ∈ S

0, j ∈ Cm k±Ï, i ∈ C0 ½ i ∈ Cl , l 6= mØ3, j ∈ Cm k±Ï, i ∈ D ½ i ∈ Cm

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

4©Ù:

� i , j ∈ S , b�4 πj = limn→∞ p(n)ij 3Ø6 i .

eπj > 0, ∀ i ∈ S ,

∑j∈S

πj = 1,

K¡ (πj ; j ∈ S) ´T Markov ó�4©Ù.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

H{ó�4©Ù:

� Markov ó´H{ó (Ø�G�Ñ´H{�), KÙ4©Ù:

πj =1

µjj, ∀ j ∈ S

e¡5§|�):

xj =∑i∈S

xipij , xj > 0, ∀ j ∈ S ;∑j∈S

xj = 1.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y:

(a). d±c=£VÇ4o(:

πj = limn→∞

p(n)ij =

1

µjj> 0, ∀ j ∈ S

(b). d C -K §: p(n+1)ij =

∑k∈S p

(n)ik pkj . d Fatou Ún: ∀ j ∈ S ,

πj = limn→∞p(n+1)ij ≥

∑k∈S

limn→∞p(n)ik pkj =

∑k∈S

πkpkj

¢Sþ, þª=�Ò¤á

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c). y�ª πj =∑

k∈S πkpkj , ∀ j ∈ S :

^y{, b�3 j0 ∈ S ¦ (b) ¥Ø�ª= ’>’ ¤á, K

∑j∈S

πj >∑j∈S

(∑k∈S

πkpkj

)=∑k∈S

∑j∈S

pkj

πk =∑k∈S

πk ,

�)gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(d). y�ª∑

j∈S πj = 1:

^ (c) �ª¿S^ C -K §:

πj =∑k∈S

πkpkj =∑k∈S

(∑i∈S

πipik

)pkj = · · · =

∑i∈S

πip(n)ij

du

1 = limn→∞∑j∈S

p(n)ij ≥

∑j∈S

limn→∞p(n)ij =

∑j∈S

πj

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(e). ^ (d) �Ø�ª:∑i∈S

∣∣∣πip(n)ij ∣∣∣ ≤∑i∈S

πi ≤ 1

ù�U^ DCT �:

πj = limn→∞

∑i∈S

πip(n)ij =

∑i∈S

(lim

n→∞p(n)ij

)πi = πj

∑i∈S

πi

du πj > 0, Kk∑

i∈S πi = 1

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(f). §|)5:

� (π′j ; j ∈ S) ´5§|,), = π′j =∑

k∈S π′kpkj .

aqþãy²§lk

π′j =∑i∈S

π′ip(n)ij

- n→∞, ¿^ DCT �:

π′j =

(∑i∈S

π′i

)πj = πj

Ïd5§|)´

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

~K:

Markov X G�m S = {0, 1, 2} , Ú=£VÇÝ:

P =

0.5 0.4 0.10.3 0.4 0.30.2 0.3 0.5

,∀ i , j ∈ S , O limn→∞ p(n)ij Ú µjj .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

):

(a). xG�=£ã: T Markov ´H{ó

(b). )5§|:

πj =2∑

i=0

πipij , πj > 0; j = 0, 1, 2,2∑

j=0

πj = 1

u´

π0 =21

62, π1 =

23

62, π2 =

18

62

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

Markov ó�²©Ù:

¡VÇ©Ù (πj ; j ∈ S) ´ Markov ó X �²©Ù, XJ§÷v:

πj =∑i∈S

πipij , ∀ j ∈ S

ePþ π = (πj ; j ∈ S), Kþª�duµ

π = πP = πP(n) = πPn, ∀ n ∈ N

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

²©Ù�)º:

e� Markov ó�Щ©Ù²©Ù π, K

(a). ∀ n ∈ N, ýé©Ù P(Xn = i) = πi , i ∈ S

(b). Markov ó´î²�

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (a):

^²©Ù�½Â:

P(Xn = i) = P {∪k∈S(X0 = k),Xn = i}=

∑k∈S

P(X0 = k)P(Xn = i | X0 = k)

=∑k∈S

πkp(n)ki = πi

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (b):

∀ t1, . . . , tn 9 m, Ú i1, . . . , in ∈ S ,

P(Xt1+m = i1,Xt2+m = i2, . . . ,Xtn+m = in)= P {∪i0∈S(X0 = i0,Xt1+m = i1,Xt2+m = i2, · · · ,Xtn+m = in)}=∑i0∈S

P(X0 = i0,Xt1+m = i1,Xt2+m = i2, · · · ,Xtn+m = in)

=∑i0∈S

πi0p(t1+m)i0i1

p(t2−t1)i1i2

· · · p(tn−tn−1)in−1in

= πi1p(t2−t1)i1i2

· · · p(tn−tn−1)in−1in= P(Xt1 = i1,Xt2 = i2, . . . ,Xtn = in)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

Ún:

∀ i ∈ S , K

limn→∞

1

n

n∑m=1

p(m)ij =

{0, e j ~½"~,fijµjj, e j �~

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

Úny²:

(a). ^�e¡ê�4�(J:

ê� (an; n ≥ 1) ÷v: 3��ê d ¦

limm→∞

amd+r = br , r = 1, 2, . . . , d ,

K

limn→∞

1

n

n∑m=1

am =1

d

d∑r=1

br

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(b). G� j ~½"~:

d limm→∞ p(m)ij = 0, ∀ i ∈ S . 3þ¡�ê�(J¥

� am = p(m)ij Ú d = 1, KUk r = 1 b1 = 0. u´

limn→∞

1

n

n∑m=1

p(m)ij = b1 = 0

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c). G� j �~±Ï dj : d±c4(J:

limm→∞

p(mdj+r)ij = fij(r)

djµjj, r = 1, 2, . . . , dj

3þ¡ê�¥� am = p(m)ij Ú d = dj , K br = fij(r)

djµjj, �

limn→∞

1

n

n∑m=1

p(m)ij =

1

dj

dj∑r=1

br =1

dj

dj∑r=1

fij(r)djµjj

=fijµjj.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

¹�~��Ø� Markov ó�²©Ù:

� X ´k�~��Ø� Markov ó ()H{ó), K

πj =1

µjj, j ∈ S

´ Markov ó X �²©Ù.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y:

� X ´k�~��Ø� Markov ó ()H{ó), K

πj =1

µjj, j ∈ S

´ Markov ó X �²©Ù.

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(a). =y²: π = πP:

∀ i , j ∈ S , ^ C -K §:

1

n

n∑m=1

p(m+1)ij =

1

n

n∑m=1

(∑k∈S

p(m)ik pkj

)=∑k∈S

(1

n

n∑m=1

p(m)ik

)pkj

- n→∞, ^ Fatou Ún:

πj ≥∑k∈S

(limn→∞

1

n

n∑m=1

p(m)ik

)=∑k∈S

πkpkj

= πj ≥∑

k∈S πkpkj , j ∈ S .

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(b). ^y{:

πj =∑k∈S

πkpkj , j ∈ S

(c). g: XÛy² ( 1µjj ; j ∈ S) ´VÇ©Ù (5: ±Ï d = 1 , d(Ø®y²)

(d). 5e(Ø (5: ±Ï d = 1 , 5®y²)

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

Markov ó X �²©Ù:

?¿ Markov ó�G�mþ©):

S = D ∪ C0 ∪ H

Ù¥ C0 "~Ø�48, H = ∪k≥1Ck ´�~Ø�48 Ck (k = 1, 2, . . . ) ¿, K

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(a). X Ø3²©Ù ⇐=⇒ H = ∪k≥1Ck = ∅

(b). X 3²©Ù ⇐=⇒ X k�~�Ø�48

(c). X 3áõ²©Ù ⇐=⇒ X �3ü±þ�~��Ø�48

(d). kG� Markov óo3²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (a)

(a1). � H = ∪k≥1Ck = ∅. ^y{, b� X k²©Ù π 6= 0, K

π = πPn

- n→∞, ¿5¿�: H = ∅ , Pn → 0 (n→∞) � π = 0, π 6= 0 gñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(a2). � X Ø3²©Ù. ^y{, Øb� H = C1.� P1 ´ X =£VÇÝ P ¥éAuG�8 C1 �fÝ,K3 C1 þ�²©Ù π1 π1 = π1P1. y P ©¬L«¦:

P =

(P1 0R1 R2

),

� π = (π1, 0), K

πP = (π1, 0)

(P1 0R1 R2

)= (π1P1, 0) = (π1, 0) = π,

� π ´ X �²©Ù, Ø3²©Ùgñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (b)

(b1). � X k�~�Ø�48, K·®²y²: X 3�²©Ù

(b2). � X k²©Ù, Kd (a) : H 6= ∅. ^y{, b� H �´üØÓ��~��Ø�48�¿, Ø� H = C1 ∪ C2, dò P ©¬L«:

P =

P1 0 00 P2 0R1 R2 R3

,Ù¥ P1,P2 ©O´ P ¥éAuG�8Ü C1,C2 �fÝ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(b3). u´3 π1 = (πk1 ; k ∈ N) Ú π2 = (πk2 ; k ∈ N) ÷v:

π1 = π1P1,∑k∈C1

πk1 = 1; π2 = π2P2,∑k∈C2

πk2 = 1

� π = (π1, 0) Ú π′ = (π2, 0), K π = πP Ú π′ = π′P. ù`

² π, π′ þ X �²©Ù, ²©Ùgñ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (c)

(c1). Ø� H = C1∪C2∪C3, KX (b2)�y²: ∃ π1, π2 Ú π3 ¦{π1 = π1P1,∑

k∈C1 πk1 = 1,

{π2 = π2P2,∑

k∈C2 πk2 = 1,

{π3 = π3P3,∑

k∈C3 πk3 = 1,

Ù¥ P1,P2,P3 ©O´ P ¥éAuG�8Ü C1,C2,C3 �fÝ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c2). y3�

π = {0, λ1π1, λ2π2, λ3π3}

Ù¥

λk ≥ 0 (k = 1, 2, 3),∑k

λk = 1

N´�y π ÷v π = πP π ´VÇ©Ù. du λk ká«�{, � X �²©Ùkáõ

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

(c3). d (a) Ú (b) �(Ø=y� (c) �75

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

lllÑÑÑmmm Markov óóó

Markov óóó½½½ÂÂÂMarkov óóóVVVÇÇÇ©©©ÙÙÙMarkov óóóGGG���©©©aaaMarkov óóóGGG���mmm©©©)))444²²²©©©ÙÙÙ

4²©Ù

y (d):

(d1). dukG�� Markov óo3�~G�8, Ïd½3²©Ù

���ááá��� ÅÅÅLLL§§§µµµ111888ÙÙÙ

离散时间 Markov 链Markov 链定义Markov 链概率分布Markov 链状态分类Markov 链状态空间分解极限与平稳分布