Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
1nÙ £8ëê��O
Tianxiao Pang
Zhejiang University
September 28, 2016
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SN
1 ���¦�O
2 ���¦�O�5�
3 �å���¦�O
4 £8�ä
5 Box-CoxC�
6 2Â���¦�O
7 õ��5
8 *�O
9 ̤©�O
10 SteinØ �O
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�O£8ëê��Ä��{´���¦{(Least SquaresMethod). �Ùcn!?ØXÛA^���¦{¦£8ëê����¦�O, ¿ïÄù«�O�Ä�5�. 31oÚ1Ê!, ·�?Ø£8�.�Ä�b��·^5±9�ùb�Ø·^�, éêâAT��C�(Box-CoxC�). 318!·�ò?Ø2Â���¦�O. 31Ô!·�ò?Ø�«AÏ�gCþ, ¿?اXÛ����¦�O�5�³. 1l!Ú1Ê!·�ò?Øü«#��O�{: *�OṲ́©�O. ���!{ü0�SteinØ �O.
lùpm©, ·�b½gCþØ´�ÅCþ, Ï�§��� �±�<���. Ó�·��½: ���gCþ^��i1L«,���ÏCþ´�ÅCþ, �^��i1L«(k��L«��*ÿ�).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
���¦�O
^yL«ÏCþ, x1, · · · , xpL«(�U)éykK��p�gCþ. b�§��m÷vXe��5'Xª:
y = β0 + β1x1 + · · ·+ βpxp + e, (3.1.1)
Ù¥e´�ÅØ�, β0, · · · , βpL«�����ëê. ¡β0�£8~ê, ¡β1, · · · , βp�£8Xê. k�r§�Ú¡�£8Xê. b½®²k��
(xi1, · · · , xip, yi), i = 1, · · · , n,
K§�÷v
yi = β0 + β1xi1 + · · ·+ βpxip + ei, i = 1, · · · , n. (3.1.2)
b�Ø��ei, i = 1, · · · , n÷vGauss-Markovb�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e^ÝL«, K���
Y =
y1
y2...yn
=
1 x11 · · · x1p
1 x21 · · · x2p...
......
...1 xn1 · · · xnp
β0
β1...βp
+
e1
e2...en
,
½��
Y = Xβ + e. (3.1.3)
Ù¥Y´n× 1��Å*ÿ�þ(ÏCþ�þ), X´n× (p+ 1)�®��OÝ(b�p+ 1 ≤ n), β´(p+ 1)× 1���ëê�þ, e´n× 1��ÅØ��þ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
òGauss-Markovb���¤Ý/ª:
E(e) = 0,Cov(e) = σ2In. (3.1.4)
ò(3.1.3)Ú(3.1.4)Ü�3�å, =����Ä���5£8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In. (3.1.5)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�^���¦�{Ïéβ��O, ù��OÏd�¡����¦�O(LSE: Least Squares Estimator). ù��{´Ïé��β��O, ¦� ��þe = Y −Xβ��Ý�²�‖Y −Xβ‖2����.
P
Q(β) = ‖Y −Xβ‖2 = (Y −Xβ)′(Y −Xβ)
= Y′Y − 2Y
′Xβ + β
′X
′Xβ.
éβ¦�, -Ù�u", ���§|
X′Xβ = X
′Y . (3.1.6)
¡ù��§|��5�§|(½�K�§|). ù��§|k��)�¿�^�´X
′X��´p+ 1, ù�duX��´p+ 1(=X
´�÷��). Ï�X ´�±<����, ¤±·�ob½X´�÷��. u´·���(3.1.6)���)
β = (X′X)−1X
′Y . (3.1.7)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
±þ�?Ø�U`²β´Q(β)���7:, ��7Ò´���:.eyβ(¢´Q(β)����:.
é?¿�β, k
‖Y −Xβ‖2 = ‖Y −Xβ +X(β − β)‖2
= ‖Y −Xβ‖2 + (β − β)′X
′X(β − β)
+2(β − β)′X
′(Y −Xβ). (3.1.8)
Ï�β÷v�5�§|(3.1.6), ¤±X′(Y −Xβ) = 0. ùy²
é?¿�β, k
‖Y −Xβ‖2 = ‖Y −Xβ‖2 + (β − β)′X
′X(β − β). (3.1.9)
duX′X´�K½Ý, ¤±(β − β)
′X
′X(β − β) ≥ 0. u´
‖Y −Xβ‖2 ≥ ‖Y −Xβ‖2.
�y.Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Pβ = (β0, β1, · · · , βp)′, ·��±��(²�)£8�§:
Y = Xβ ½ y = β0 + β1x1 + · · ·+ βpxp. (3.1.10)
ù��§´Ø´£ãy�x1, · · · , xp�ý¢'X, �I�?�Ú�ÚO©Û, 3�±�?n.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.1���5£8. b�gCþ�k��, P�x. ���(xi, yi), i = 1, · · · , n. u´k�5£8�.
yi = β0 + β1xi + ei, i = 1, · · · , n.
ù���5�§|�n
n∑i=1
xi
n∑i=1
xi
n∑i=1
x2i
(β0
β1
)=
n∑i=1
yi
n∑i=1
xiyi
,
��OÝX´�÷��, =xi, i = 1, · · · , nØ����,n∑i=1
(xi − x)2 6= 0, u´β0Úβ1�LSE�
β0 = y − β1x,
β1 =
∑ni=1(xi − x)(yi − y)∑n
i=1(xi − x)2=:
SxySxx
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3£8©Û¥, ·�k�r�©êâ?1¥%zÚIOz. -
xj =1
n
n∑i=1
xij , j = 1, · · · , p.
ò(3.1.2)U��
yi = α+ β1(xi1 − x1) + · · ·+ βp(xip − xp) + ei,
i = 1, · · · , n. (3.1.11)
ùp, α = β0 + β1x1 + · · ·+ βpxp. ¡(3.1.11)�¥%z�.. P
Xc =
x11 − x1 x12 − x2 · · · x1p − xpx21 − x1 x22 − x2 · · · x2p − xp
......
......
xn1 − x1 xn2 − x2 · · · xnp − xp
. (3.1.12)
K(3.1.11)�U��
Y = α1n +Xcβ + e =(1n Xc
)(αβ
)+ e. (3.1.13)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ùp, β = (β1, · · · , βp)′. 5¿�
1′nXc = 0 (3.1.14)
Ïd�5�§|���(n 0
0 X′cXc
)(αβ
)=
(1
′nY
X′cY
), (3.1.15)
�d/�¤ {nα = 1
′nY ,
X′cXcβ = X
′cY
(3.1.16)
u´£8ëê�LSE�{α = y,
β = (X′cXc)
−1X′cY .
(3.1.17)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ïd, éu¥%z�5£8�.(3.1.11), £8~ê�LSEo´ÏCþ���þ�, £8Xêβ�LSE β = (X
′cXc)
−1X′cY�du
l�5£8�.Y = X′cβ + e¥O�β�LSE. 3¢SA^¥, O
�(X′cXc)
−1o´'O�(X ′X)−1��B�:, �·�o´AO'%£8Xê, ¤±¥%z´kÐ?�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.2���5£8(Y). ò~3.1.1¥����5£8�.?1¥%z, �
yi = α+ β1(xi − x) + ei, i = 1, · · · , n. (3.1.18)
dúª(3.1.17)�LSEα = y,
β1 =
∑ni=1(xi − x)yi∑ni=1(xi − x)2
=
∑ni=1(xi − x)(yi − y)∑n
i=1(xi − x)2.
(3.1.19)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
,, ·���±égCþ�IOz?n. P
s2j =
n∑i=1
(xij − xj)2, j = 1, · · · , p,
zij =xij − xjsj
, i = 1, · · · , n; j = 1, · · · , p.
-Z = (zij)n×p, §äk5�:
1′nZ = 0,
R = Z′Z = (rij),
Ù¥
rij =
∑nk=1(xki − xi)(xkj − xj)
sisj, i, j = 1, · · · , p (3.1.20)
�gCþxi�xj����'Xê. ¤±R´gCþ��'XêÝ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
égCþ?1IOz�Ð?: (1)�±^5©Û£8gCþ�m��''X; (2)IOz�Øþj�K�, Bué£8Xê�O��ÚO©Û.
²LIOz���5£8�.�
yi = α+xi1 − x1
s1β1 + · · ·+ xip − xp
spβp + ei, (3.1.21)
½�¤Ý/ª
Y = α1n +Zβ + e =(1n Z
)(αβ
)+ e.
��ëê�LSE�
α = y, β = (β1, · · · , βp)′
= (Z′Z)−1Z
′Y .
£8�§�
y = α+x1 − x1
s1β1 + · · ·+ xp − xp
spβp. (3.1.22)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.1.3��Á�Nì��ðøA9þ, ¦Ù�±ð§, eL¥,gCþxL«Nì±��íü �m�²þ§Ý(◦C), yL«ü �mS�Ñ��ðþ(L), �*ÿ25 ��mü . ã3.1.1´ùêâ�Ñ:ã, éù|êâ, A^¥%z�5£8�.(3.1.18),·���
y = 9.424, x = 52.6,
£8ëê�LSE�
α = y = 9.424, β1 = −0.0798.
¤±£8�§�
y = 9.424− 0.0798(x− 52.6),
½�¤y = 13.623− 0.0798x.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
L3.1.1: �ðêâ
SÒ y(L) x(◦C) SÒ y(L) x(◦C)1 10.98 35.3 14 9.57 39.12 11.13 29.7 15 10.94 46.83 12.51 30.8 16 9.58 48.54 8.40 58.8 17 10.09 59.35 9.27 61.4 18 8.11 706 8.73 71.3 19 6.83 707 6.36 74.4 20 8.88 74.58 8.5 76.7 21 7.68 72.19 7.82 70.7 22 8.47 58.1
10 9.14 57.5 23 8.86 44.611 8.24 46.4 24 10.36 33.412 12.19 28.9 25 11.08 28.613 11.88 28.1
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ã3.1.1 Ñ:ã
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p33 data.txt”)y=yx[, 1]x=yx[, 2]mydata=data.frame(y,x)plot(y∼x)lm.sol=lm(y∼x,data=mydata)abline(lm.sol)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
����£8�§:
y = 13.623− 0.0798x.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
���¦�O�5�
���¦�O(LSE)äk�ûÐ�5�:
½n (3.2.1)
éu�5£8�.(3.1.5), LSE β = (X′X)−1X
′Yäke�5�:
(a) E(β) = β;(b) Cov(β) = σ2(X
′X)−1.
y²: (a)´�E(Y ) = Xβ, ¤±
E(β) = (X′X)−1X
′ · E(Y ) = β.
(b)Ï�Cov(Y ) = Cov(e) = σ2In, ¤±
Cov(β) = Cov((X′X)−1X
′Y )
= (X′X)−1X
′Cov(Y )X(X
′X)−1
= (X′X)−1X
′σ2InX(X
′X)−1 = σ2(X
′X)−1.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�c´p+ 1��~ê�þ, éu�5¼êc′β(ù´����ëê),
·�¡c′β�c
′β�LSE.
íØ (3.2.1)
(a) E(c′β) = c
′β;
(b) Cov(c′β) = σ2c
′(X
′X)−1c.
=é?¿��5¼êc′β, c
′β�c
′β�à �O, ���
σ2c′(X
′X)−1c. Ï�c
′β = c
′(X
′X)−1X
′Y�y1, · · · , yn��5
¼ê, ¤±c′β�c
′β����5à �O(�5�O��´*
ÿy1, · · · , yn��5¼ê). ·���±�EÑc′β�Ù§�5Ã
�O. ù�¤c′β��5à �Oa.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.2.2, Gauss-Markov)
éu�5£8�.(3.1.5), 3c′β�¤k�5à �O¥, ���
¦�Oc′β´��������5à �O(BLUE: best linear
unbiased estimator).
y²: �a′Y�c
′β����5à �O. u´é��p+ 1���
þβ,c′β = E(a
′Y ) = a
′Xβ,
Ïd
a′X = c
′. (3.2.1)
Ï�Var(a′Y ) = σ2a
′a = σ2‖a‖2, ·�é‖a‖2�©):
‖a‖2 = ‖a−X(X′X)−1c+X(X
′X)−1c‖2
= ‖a−X(X′X)−1c‖2 + ‖X(X
′X)−1c‖2
+2c′(X
′X)−1X
′(a−X(X
′X)−1c). (3.2.2)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
d(3.2.1)�í�
2c′(X
′X)−1X
′(a−X(X
′X)−1c) = 2c
′(X
′X)−1(X
′a−c′) = 0.
díØ3.2.1(b)�í�
σ2‖X(X′X)−1c‖2
= σ2c′(X
′X)−1X
′X(X
′X)−1c
= σ2c′(X
′X)−1c
= Var(c′β).
u´, ·�y²éc′β�?���5à �Oa
′Yk
Var(a′Y ) = Var(c
′β) + σ2‖a−X(X
′X)−1c‖2
≥ Var(c′β), (3.2.3)
�Ò¤á��=�‖a−X(X′X)−1c‖ = 0, =a = X(X
′X)−1c,
d�a′Y = c
′β. ½n�y.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3�5£8�.(3.1.5)¥�k��ëêσ2(Ø���), §�N�.Ø����, 3£8©Û¥åX���^. ·�y35�Oσ2.
e = Y −Xβ´Ø��þ, Ø�*ÿ. ^β�Oβ, ¡
e = Y −Xβ = Y − Y (3.2.4)
�í�(residual)�þ. Px′i��OÝX�1i1, K
ei = yi − x′iβ, i = 1, · · · , n (3.2.5)
�1ig*ÿ�í�. g,/, ·��±òew�e����O. ·�ò^
RSS = e′e =
n∑i=1
e2i (3.2.6)
5�Eσ2�à �Oþ.Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
RSS=Residual Sum of Squares, L«í�²�Ú. §�N¢Sêâ�nØ�.(3.1.5)� l§Ý½ö`[ܧÝ. RSS��L«êâ��.[Ü��Ð.
½n (3.2.3)
(a) RSS = Y′(In −X(X
′X)−1X
′)Y =: Y
′(In −H)Y ;
(b) σ2 = RSS/(n− p− 1)´σ2�à �Oþ.
5: ¡H = X(X′X)−1X
′�lf(hat)Ý, §´��é¡��
Ý.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
y²: (a)
RSS = e′e = (Y −Xβ)
′(Y −Xβ)
= [(In −X(X′X)−1X
′)Y ]
′[(In −X(X
′X)−1X
′)Y ]
= Y′(In −X(X
′X)−1X
′)Y .
(b)dE(Y ) = Xβ,Cov(Y ) = σ2In±9½n2.2.1�
E(RSS) = E[Y′(In −X(X
′X)−1X
′)Y ]
= β′X
′(In −X(X
′X)−1X
′)Xβ
+σ2tr(In −X(X′X)−1X
′)
= σ2[n− tr(X(X′X)−1X
′)].
�â,�5�tr(AB) = tr(BA)�
tr(X(X′X)−1X
′) = tr((X
′X)−1X
′X) = tr(Ip+1) = p+ 1.
u´E(RSS) = σ2(n− p− 1).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
XJb�Ø��þeÑl��©Ù, =e ∼ N(0, σ2In), @o·��±��βÚσ2�õ��5�.
½n (3.2.4)
éu�5£8�.(3.1.5), eØ��þe ∼ N(0, σ2In), K(a) β ∼ N(β, σ2(X
′X)−1);
(b) RSS/σ2 ∼ χ2(n− p− 1);(c) β�RSS�pÕá.
y²: (a) 5¿�Y ∼ N(Xβ, σ2In)±9β = (X′X)−1X
′Y´Y
��5C�, @od½n2.3.4B�í�(a).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(b) �â½Â
RSS = Y′(In −H)Y = Y
′(In −H)Y = Y
′NY ,
ùpN = In −H, §´��é¡��Ý. 5¿�NX = 0, ¤±
RSS = (Xβ + e)′N(Xβ + e) = e
′Ne. (3.2.7)
5¿�e ∼ N(0, σ2In), ¤±�â½n2.4.3·��Iy²N���n− p− 1. |^��Ý���u§�,ù�5�, ·���
rk(N) = tr(In −X(X′X)−1X
′)
= n− tr(X(X′X)−1X
′)
= n− tr((X′X)−1X
′X)
= n− p− 1.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(c) Ï�β = β + (X′X)−1X
′e, RSS = e
′Ne, 5¿�
(X′X)−1X
′N = 0,
¤±d½n2.4.5��(X′X)−1X
′e�RSS�pÕá, =β�RSS�
pÕá.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�β�1��©þ�~ê�β0�, �c = (0, · · · , 0, 1, 0, · · · , 0)′, Ù
¥1 3c�1i+ 1� �, Kc′β = βi. 2Pβ = (β0, · · · , βp)
′, u
´c′β = βi. 2^(A)iiL«ÝA�1(i, i)��, @o·Xeí
Ø:
íØ (3.2.2)
éu�5£8�.(3.1.5), ee ∼ N(0, σ2In), K(a) βi ∼ N(βi, σ
2[(X′X)−1]i+1,i+1);
(b)3βi, i = 1, · · · , p����5à �O¥, βi, i = 1, · · · , p´������ö.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ò½n3.2.1Ú½n3.2.4A^u¥%z�.(3.1.13), Kk
íØ (3.2.3)
éu¥%z�.(3.1.13), 5¿ùp�β = (β1, · · · , βp)′, k
(a) E(α) = α,E(β) = β, ùpα = y, β = (X′cXc)
−1X′cY ;
(b)
Cov
(α
β
)= σ2
(1n 0
0 (X′cXc)
−1
);
(c)e?�Úb�e ∼ N(0, σ2In), K
α ∼ N(α,σ2
n), β ∼ N(β, σ2(X
′cXc)
−1),
�α�β�pÕá.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
R2 =ESS
TSS, (3.2.8)
Ù¥
ESS =
n∑i=1
(y − yi)2 = (Y − y1n)′(Y − y1n)
¡�£8²�Ú(½)º²�Ú: Explained Sum of Squares),
TSS =
n∑i=1
(yi − y)2 = (Y − y1n)′(Y − y1n)
¡�o �²�Ú(½¡�o²�Ú: Total Sum of Squares).¡R2��½Xê½ÿ½Xê, ¡R =
√R2�E�'Xê.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
d�5�§|�±y²
n∑i=1
(yi − y)2
=
n∑i=1
(yi − yi + yi − y)2
=
n∑i=1
(yi − yi)2 +
n∑i=1
(yi − y)2 + 2
n∑i=1
(yi − yi)(yi − y)
=n∑i=1
(yi − yi)2 +
n∑i=1
(yi − y)2.
=TSS = ESS + RSS.
ÏdR2Ýþ£8gCþx1, · · · , xpéÏCþy�[ܧÝ�Ð�. 0 ≤ R2 ≤ 1, §����, L²y�ÃgCþk�����'X.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e¦^¥%z�.(3.1.13), @oESS�ÏLe�úªO�:
ESS = β′X
′cY = Y
′Xc(X
′cXc)
−1X′cY ,
ùpβ = (β1, · · · , βp)′. ¯¢þ, dY = α1n +Xcβ9úª
(3.1.17)��Y − y1n = Y − α1n = Xcβ.
¤±
ESS = (Y − y1n)′(Y − y1n) = (Xcβ)
′Xcβ
= β′X
′c ·Xc(X
′cXc)
−1X′cY
= β′X
′cY .
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éu���5£8�.
yi = β0 + β1xi + ei, i = 1, · · · , n.
�±y²R2 = r2, Ù¥r��.¥gCþ�ÏCþ����'Xê.
¯¢þ, e·�r�.¥%z:
yi = α+ β1(xi − x) + ei, i = 1, · · · , n,
@o��
β1 =
∑ni=1(xi − x)yi∑ni=1(xi − x)2
,
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
l
ESS = β1
n∑i=1
(xi − x)yi =
[∑ni=1(xi − x)yi
]2∑ni=1(xi − x)2
=
[∑ni=1(xi − x)(yi − y)
]2∑ni=1(xi − x)2
.
¤±
R2 =ESS
TSS=
[∑ni=1(xi − x)(yi − y)
]2∑ni=1(xi − x)2
∑ni=1(yi − y)2
= r2.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.2.1�â²��, 3<��p���^�e, ÙÉØ� Øy�Nx1!c#x2k'. yÂ813¶If�ÿþêâ, �eL. Áïáy'ux1, x2��5£8�§.
L3.2.1: ÉØêâ
SÒ xi1 xi2 yi SÒ xi1 xi2 yi1 152 50 120 8 158 50 1252 183 20 141 9 170 40 1323 171 20 124 10 153 55 1234 165 30 126 11 164 40 1325 158 30 117 12 190 40 1556 161 50 125 13 185 20 1477 149 60 123
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
|^¥%z�.
yi = α+ β1(xi1 − x1) + β2(xi2 − x2) + ei, i = 1, · · · , 13.
²O���
x1 =1
13
13∑i=1
xi1 = 166.8, x2 =1
13
13∑i=1
xi2 = 38.85, y =1
13
13∑i=1
yi = 130,
Xc =
−14.08 11.1516.92 −18.854.92 −18.85−1.08 −8.85−8.08 −8.85−5.08 11.15−17.08 21.15−8.08 11.153.92 1.15−13.08 16.15−2.08 1.1523.92 1.1518.92 −18.85
,
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�5�§|X′cXcβ = X
′cY�{
2078.92β1 − 1533.85β2 = 1607,−1533.85β1 + 2307.69β2 = −715.
)�β1 = 1.068, β2 = 0.4, α = y = 130. ¤±£8�§�
y = α+ β1(x1 − x1) + β2(x2 − x2)
= 130 + 1.068× (x1 − 166.8) + 0.4× (x2 − 38.85)
= −62.963 + 1.068x1 + 0.4x2.
d, ����
ESS = β′X
′cY = 1430.276, TSS = 1512,
¤±R2 = 1430.276/1512 = 0.9459.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p39 data.txt”)x1=yx[, 1]x2=yx[, 2]y=yx[, 3]mydata=data.frame(y,x1,x2)lm.sol=lm(y∼x1+x2,data=mydata)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
dR$1(J�: y = −62.963 + 1.068x1 + 0.4x2, R2 = 0.9461.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�å���¦�O
éβØ\?Û�å^���/e, ·�?ا�LSE±9§�Ä�5�. �3�AÏ|Ü, ~Xb�u�¯K, ·�I�¦�k�½�å^��LSE.
b�
Aβ = b (3.3.1)
´���N�5�§|(=�§|k)), Ù¥A´k × (p+ 1)®�Ý, ��k, b´k × 1®��þ. ·�^Lagrange¦f{¦�5£8�.(3.1.5)3÷v�5�å(3.3.1)��LSE.=3(3.3.1)ù�^�e¦β¦Q(β) = ‖Y −Xβ‖2����.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�E9ϼê
F (β,λ) = ‖Y −Xβ‖2 + 2λ′(Aβ − b)
= (Y −Xβ)′(Y −Xβ) + 2λ
′(Aβ − b),
Ù¥λ = (λ1, · · · , λk)′�Lagrange¦f�þ. é¼êF (β,λ)'
uβ¦�¿-§�u0, �
−X ′Y +X
′Xβ +A
′λ = 0. (3.3.2)
y3, ·�I�)d(3.3.1)Ú(3.3.2)|¤�éá�§|.
·�^βcÚλcL«ù��§|�). d(3.3.2)�
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
βc = (X′X)−1X
′Y − (X
′X)−1A
′λc
= β − (X′X)−1A
′λc. (3.3.3)
�\(3.3.1)�
b = Aβc = Aβ −A(X′X)−1A
′λc,
��d/�¤
A(X′X)−1A
′λc = Aβ − b. (3.3.4)
duA���k, ¤±A(X′X)−1A
′´k × k�_Ý, Ïd(3.3.4)
k��)λc = (A(X
′X)−1A
′)−1(Aβ − b),
2ò§�\(3.3.3)�
βc = β − (X′X)−1A
′(A(X
′X)−1A
′)−1(Aβ − b). (3.3.5)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
eyβc(¢´�5�åAβ = beβ�LSE. ù�Iy²(a) Aβc = b;(b)é��÷vAβ = b�β, Ñk‖Y −Xβ‖2 ≥ ‖Y −Xβc‖2.
d(3.3.5)�í�(a). �y(b), ò‖Y −Xβ‖2©)�
‖Y −Xβ‖2
= ‖Y −Xβ‖2 + ‖X(β − β)‖2
= ‖Y −Xβ‖2 + ‖X(β − βc + βc − β)‖2
= ‖Y −Xβ‖2 + ‖X(β − βc)‖2 + ‖X(βc − β)‖2,(3.3.6)
ùp�í�^�eã'X: (Y −Xβ)′X = 0±9é��÷
vAβ = b�β,
(β − βc)′X
′X(βc − β) = λ
′cA(βc − β) = λ
′c(Aβc −Aβ) = 0.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(3.3.6)L²: é��÷vAβ = b�β, ok
‖Y −Xβ‖2 ≥ ‖Y −Xβ‖2 + ‖X(β − βc)‖2, (3.3.7)
�Ò¤á��=�X(βc − β) = 0, =β = βc(Ï�X�÷�). Ïd3(3.3.7)¥^βc�Oβ, �Ò¤á. =
‖Y −Xβc‖2 = ‖Y −Xβ‖2 + ‖X(β − βc)‖2. (3.3.8)
y3, (Ü(3.3.7)Ú(3.3.8)B�í�(b).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�rβc¡�β��å���¦�O, u´ke��½n:
½n (3.3.1)
éu�5£8�.(3.1.5), ÷v(3.3.1)��åLSE�
βc = β − (X′X)−1A
′(A(X
′X)−1A
′)−1(Aβ − b),
Ù¥β = (X′X)−1X
′Y´Ã�å^�e�LSE.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.3.13U©ÿþ¥, éU�¥�n�( :�¤�n�/ABC�n�S�θ1, θ2, θ3?1ÿþ, ���y1, y2, y3, du�3ÿþØ�, ¤±I�éθ1, θ2, θ3?1�O, ·�|^�5�.L«k'�þ
y1 = θ1 + e1,y2 = θ2 + e2,y3 = θ3 + e3,θ1 + θ2 + θ3 = π.
�¤Ý/ª {Y = Xβ + e,Aβ = b,
ùpY = (y1, y2, y3)′, β = (θ1, θ2, θ3)
′, X = I3, A = (1, 1, 1),
b = π. 5¿�β = (X′X)−1X
′Y = Y , A^½n3.3.1, ²O��
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
βc =
y1
y2
y3
− 1
3(
3∑i=1
yi − π)
111
,
=
θi = yi −1
3(
3∑i=1
yi − π), i = 1, 2, 3
�θi��åLSE.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
£8�ä
éu�5£8�.(3.1.5):
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In,
·�3?Ø��ëê����¦�O±9�Oþ�©Ù�, �Xe��Ä��b�:
(a)�5b�: ÏCþ�gCþäk�5�''X;(b)��à5b�: Var(ei) = σ2, i = 1, · · · , n;(c)Ø�'5b�: Cov(ei, ej) = 0, i 6= j.(d)��5b�: ei ∼ N(0, σ2), i = 1, · · · , n.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
XJùb�ؤá, @o·�±c?Ø����¦�O±9§�ÚO5�Òk�U´Ø¤á�. Ïd, 3¢S¯K¥, �·�k�1êâ�, I��·��êâ´Ä÷v½öÄ�÷vùb�. ù´�.�ä�SN.
Ï�ùb�Ñ��ÅØ�ek', í��þe�w¤´e����O, Ïd·��±ÏLí�5©Û4�Ä�b�´Ä¤á. �Ï�ù��Ï, ùÜ©SN��¡�í�©Û.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ØI�é�.?1�ä, ·��I�éêâ��?1�ä, §�¹É~:�äÚrK�:�ä.
3£8©Û¥, ¤¢É~:(q�l+:)´�éQ½�. lé��êâ:. �§e��O(�½Â´��(J�, �8�vkÚ��½Â. 8c, éÉ~:��61�w{´: (1)É~:´�@�ý�õêêâ:²wØ�N�êâ:. ù�É~:�n)�¤b½©Ù¥�4à:;(2)É~:Ò´@À/:, =´��ý�õêêâ:Ø´5gÓ�©Ù��Oêâ:.
É~:�·\òéëê��OE¤K�, Ïd·�I�uÿÑÉ~:¿ò§íØ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
êâ8¥�rK�:´�éÚOíä(ëê�O!b�u��)�)��K��êâ:, 3d�§¥, A�é£8ëêβ����¦�Ok��K��êâ:.
éz�|êâ:(x′i, yi), ·�F"§é£8ëê��Ok�½�
K�, �qF"ù«K�ØU��. ù�, ·����²�£8�§Òk�½�½5. ÄK, XJ�O�ü|êâé�OkÉ~��K�, �·�GØùêâ��, ÒU����5�Éé��²�£8�§.
Ïd3£8©Û¥, ·�I��z|êâéëê�O�K���. ùÜ©SN�¡�K�©Û.
uÿÑrK�:�, ·�I�Ø�ùêâ´Ä�~, eØ�~KíØ�, ÄK�ÄÂ8�õ�êâ½æ^�è�O�{± �/DºrK�:é�O�K�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
É~:ÚrK�:´ü�ØÓ�Vg. l�¡�©Û�±wÑ,§��mQk�½�éX�k�½�«O.
rK�:�UÓ�´É~:��UØ´, ��, É~:�UÓ�q´rK�:��UØ´.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ïd, £8�ä�¹êâ��ä��.��äùüÜ©SN. Ù¥êâ��ä�)É~:�äÚrK�:�ä, �.��ä�¹�5�ä!��à5�ä!Ø�'5�äÚ��5�ä4 Ü©SN.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
k5?Øí�©Û:
éu�5£8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In, (3.4.1)
·�^
ei = yi − x′iβ, i = 1, · · · , n (3.4.2)
L«1ig*ÿ�í�, ¿r§w¤´ei����O. e�.(3.4.1)�(, @oeiAäkei��5G.
PY = Xβ, ¡Y�[Ü��þ, ¡yi = x′iβ�1i�[Ü�. ´�
Y = X(X′X)−1X
′Y = HY , (3.4.3)
Ù¥H = X(X′X)−1X
′�¡�lfÝ, §´é¡��Ý.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
í�e��L«�
e = Y − Y = (I −H)Y = (I −H)e. (3.4.4)
I −H�´��é¡��Ý.
½n (3.4.1)
e(3.4.1)¤á, K(a) E(e) = 0, Cov(e) = σ2(I −H);(b) Cov(Y , e) = 0.(c)e?�Úb�e ∼ N(0, σ2I), K
e ∼ N(0, σ2(I −H)).
y²: N´, Ñ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Var(ei) = σ2(1− hii)(hiiL«ÝH�1i�é����), �à5, ùkNuei�¢SA^. Ïd·��Ĥ¢�Æ)zí�
ri =ei
σ√
1− hii, i = 1, · · · , n, (3.4.5)
ùpσ2 = RSS/(n− p− 1) = e′e/(n− p− 1).
=¦3e ∼ N(0, σ2I)�^�e, ri�©ÙE,'�E,, ��Cq/@�ri�pÕá�ÑlN(0, 1)(��FW(1987)). d½n3.4.1��{ri, i ≥ 1}�{yi, i ≥ 1}Õá({ei, i ≥ 1}�{yi, i ≥ 1}�Õá).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
í�ã´±,«í�(Æ)zí�ri½ÊÏí�ei)�p�I, ±?ÛÙ§�þ�î�I�Ñ:ã. c®�Ñí���Ø�ei��OAT�ei��Ø�, ��âí�ã5G´Ä�Ak�5����,Ò�±é�.b��Ün5Jø�k^�&E.
e¡·�±[Ü�yi�î�I, Æ)zí�ri�p�I�í�ã�~?Øí�ã�äNA^. ���J�´, Ï~�¹e, ±ÊÏí�ei�p�IÚ±Æ)zí�ri�p�I�í�ã/G���Ó, ±,�gCþxj�î�I½ö±SÒi�î�IÚ±[Ü�yi�î�I�í�ã/G����Ó.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�5�ä : e�5b�¤á, @oeiØ�¹5ggCþ�?Û&E, Ïdí�ãØA¥y?Ûk5K�/G, ÄKknd~¦�5b�ؤá.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��à5�ä : e��à5, @oí�ãþ�:´/þ!0ÑÙ�, ÄK, í�ãÏ~¬¥y//i.0½/�/i.0½üöo k��/G.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ø�'5�ä : eÕá5¤á, @oí�ãþ�:Ø¥y5K5, ÄK, Ñ:ãò¥y”8ì5”½/ì���50�/G.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��5�ä : e��5¤á, @oÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1). ¤±, 3±ri�p�Iyi�î�I�í�ãþ, ²¡þ�:(yi, ri), i = 1, · · · , n ��Aá3°Ý�4�Y²�|ri| ≤ 2«�S(ù�ªÇA395%�m), �Ø¥y?Ûª³.
·���±^Æ)zí��QQ ã5���5�ä. �|Nþ�n�êâ'u,�©ÙF (x)�QQãÒ´±êâ�i/n© ê�p�I, ±F (x)�i/n© ê�î�I�Ñ:ã.
XJêâ´5gF (x)���{ü�Å��, KùÑ:A��3�^��þ,ÏdÏL�äQQã´ÄCq3�^��þ���¤��êâ8´ÄÑl¤�½©Ù��ÚO�â.
d, ·���±Shapiro-Wilk�{5���5u�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��5�ä : e��5¤á, @oÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1). ¤±, 3±ri�p�Iyi�î�I�í�ãþ, ²¡þ�:(yi, ri), i = 1, · · · , n ��Aá3°Ý�4�Y²�|ri| ≤ 2«�S(ù�ªÇA395%�m), �Ø¥y?Ûª³.
·���±^Æ)zí��QQ ã5���5�ä. �|Nþ�n�êâ'u,�©ÙF (x)�QQãÒ´±êâ�i/n© ê�p�I, ±F (x)�i/n© ê�î�I�Ñ:ã.XJêâ´5gF (x)���{ü�Å��, KùÑ:A��3�^��þ,ÏdÏL�äQQã´ÄCq3�^��þ���¤��êâ8´ÄÑl¤�½©Ù��ÚO�â.
d, ·���±Shapiro-Wilk�{5���5u�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−2
−1
01
2
y
r
©Û: Ù5G���5b�Ä���, Ïd�@���5b�e ∼ N(0, σ2I)´Ün�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−1.
5−
1.0
−0.
50.
00.
51.
0
y
r
©Û: í�ã¥y/i., Ïd@���à5b�ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−6
−4
−2
02
46
y
r
©Û: í�ã¥y�/i., Ïd@���à5b�ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−0.
15−
0.10
−0.
050.
000.
050.
10
y
r
©Û: ã/¥yk5K�/G, Ïd@��5b�ØÜn, í�¥A¹kgCþ�&E.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−3
−2
−1
01
23
y
r
©Û: í�ã¥y8ì5, Ïd@�Ø�'5ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
0.0 0.2 0.4 0.6 0.8 1.0
−1.
0−
0.5
0.0
0.5
1.0
y
r
©Û: í�ã¥yì���5, Ïd@�Ø�'5ØÜn.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
lí�ã�äÑ5�U�/;¾0, �Ò´,b�^�ؤá,·�I�é¯K/éwe�0:
XJkwG¦·�~¦�5b�ؤá, @o·��±�Ä3£8gCþ¥O\,gCþ��g�, Xx2
1, x22½���x1x2�,
äNO\=gCþ�=�, IÀ¢S�J ½;
XJkwG¦·�~¦��à5b�ؤá, @o�±�ÄéÏCþ�·��C�¦#CþäkCq�����(��½5C�), ½öæ^2Â���¦�O;
XJkwG¦·�~¦Ø�'5b�ؤá, @o�±�ÄéÏCþ�/�©0, ¦#CþäkCqÕá5;
XJkwG¦·�~¦��5b�ؤá, @·��±�ÄéÏCþ���5C�–Box-CoxC�. ���J�´, Box-CoxC�´�«nÜ£n�Y, e�!2�[0�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��½5C� :
��ÅCþY�þ��µ, ���σ2, b����þ��¼ê, =b�σ2 = g(µ), Ù¥g´��®��¼ê. ~X, eY ∼ B(n, p),KE(Y ) = np, Var(Y ) = np(1− p) = µ(1− µ/n). yI�{���C�Z = f(Y ), ¦�Var(Z)�~ê. ùI�éѼêf�L�ª. (ùp, ·�^��i1YÚZL«�ÅCþ, ��i1yÚzL«��Å�Cþ)
Pz = f(y), ¿-§3y = µ?TaylorÐm, �Cqª
f(y) = f(µ) + f′(µ)(y − µ).
òyU¤�ÅCþ, k
Z = f(Y ) = f(µ) + f′(µ)(Y − µ),
Ù��c = Var(Z) = [f′(µ)]2Var(Y ) = [f
′(µ)]2g(µ). �f
′(µ) =√
c/g(µ), K��
f(y) =
∫ √c/g(y)dy.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
e¡´A�A~:
(1) g(µ)�µ¤�'�, Pg(µ) = aµ, K∫ √c
aydy = 2
√c
ay + c
′,
Ñ�~êØO, ��²��C�: Z =√Y .
(2)�g(µ)�µ2¤�'�, Pg(µ) = aµ2, �y > 0, K∫ √c
ay2dy =
√c
aln y + c
′.
Ñ�~êØO, ��éêC�: Z = lnY .
(3)�g(µ)�µ4¤�'�, Pg(µ) = aµ4, K∫ √c
ay4dy = −
√c
a
1
y+ c
′.
Ñ�~êØO, ���êC�: Z = 1/Y .Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3A^þ, Äklí�ãoÑ/��eσ2�µ�U�3�A«'X(=�O¼êg(·)), ,�lúª
f(y) =
∫ √c/g(y)dy
¦ÑéA�C�. éA«C�L�êâ©O����¦?n, �#�í�ã, w=�«C��í�ãÃ���à5��î. l¥ÀÑ�Ð���½5C�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.4.1�ïÄ^>p¸z���^>þy�z�o^>þx�'X, yÂ8,�53rêâ.
L3.4.1: ^>þêâ
i x y yi ei z =√y zi ei
1 679 0.790 1.669 -0.879 0.889 1.229 -0.3402 292 0.440 0.244 0.196 0.663 0.860 -0.1973 1012 0.560 2.896 -2.336 0.748 1.547 -0.7984 493 0.790 0.984 -0.194 0.889 1.052 -0.1635 582 2.700 1.312 1.388 1.643 1.137 0.5066 1156 3.640 3.426 0.214 1.908 1.684 0.2247 997 4.730 2.840 1.890 2.175 1.532 0.6438 2189 9.500 7.230 2.270 3.082 2.668 0.4149 1097 5.340 3.209 2.131 2.311 1.628 0.68310 2078 6.850 6.822 0.028 2.617 2.562 0.05511 1818 5.840 5.864 -0.024 2.417 2.315 0.10212 1700 5.210 5.430 -0.220 2.283 2.202 0.08013 747 3.250 1.920 1.330 1.803 1.294 0.50914 2030 4.430 6.645 -2.215 2.105 2.517 -0.412
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
i x y yi ei z =√y zi ei
15 1643 3.160 5.220 -2.060 1.778 2.148 -0.37016 414 0.500 0.693 -0.193 0.707 0.977 -0.27017 354 0.170 0.472 -0.302 0.412 0.920 -0.50718 1276 1.880 3.868 -1.988 1.371 1.798 -0.42719 745 0.770 1.912 -1.142 0.877 1.292 -0.41520 435 1.390 0.771 0.619 1.179 0.997 0.18221 540 0.560 1.157 -0.597 0.748 1.097 -0.34822 874 1.560 2.388 -0.828 1.249 1.415 -0.16623 1543 5.280 4.851 0.429 2.298 2.052 0.24524 1029 0.640 2.958 -2.318 0.800 1.563 -0.76325 710 4.000 1.784 2.216 2.000 1.259 0.74126 1434 0.310 4.450 -4.140 0.557 1.949 -1.39227 837 4.200 2.251 1.949 2.049 1.380 0.67028 1748 4.880 5.606 -0.726 2.209 2.248 0.03929 1381 3.480 4.255 -0.775 1.865 1.898 -0.03330 1428 7.580 4.428 3.152 2.753 1.943 0.81031 1255 2.630 3.791 -1.161 1.622 1.778 -0.15632 1777 4.990 5.713 -0.723 2.234 2.275 -0.04233 370 0.590 0.531 0.059 0.768 0.935 -0.16734 2316 8.190 7.698 0.492 2.862 2.789 0.073
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
i x y yi ei z =√y zi ei
35 1130 4.790 3.330 1.460 2.189 1.659 0.53036 463 0.510 0.874 -0.364 0.714 1.023 -0.30937 770 1.740 2.004 -0.264 1.319 1.316 0.00338 724 4.100 1.835 2.265 2.025 1.272 0.75339 808 3.940 2.144 1.796 1.985 1.352 0.63340 790 0.960 2.078 -1.118 0.980 1.335 -0.35541 783 3.290 2.052 1.238 1.814 1.328 0.48642 406 0.440 0.664 -0.224 0.663 0.969 -0.30643 1242 3.240 3.743 -0.503 1.800 1.766 0.03444 658 2.140 1.592 0.548 1.463 1.209 0.25445 1746 5.710 5.599 0.111 2.390 2.246 0.14446 468 0.640 0.892 -0.252 0.800 1.028 -0.22847 1114 1.900 3.271 -1.371 1.378 1.644 -0.26548 413 0.510 0.690 -0.180 0.714 0.976 -0.26249 1787 8.330 5.750 2.580 2.886 2.285 0.60150 3560 14.940 12.280 2.660 3.865 3.974 -0.10951 1495 5.110 4.675 0.435 2.261 2.007 0.25452 2221 3.850 7.348 -3.498 1.962 2.699 -0.73653 1526 3.930 4.789 -0.859 1.982 2.036 -0.054
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A^���¦{, �£8�§
y = −0.83130 + 0.00368x.
R§S:
yx=read.table(”ex p47 data.txt”)x=yx[, 1]y=yx[, 2]mydata=data.frame(y,x)lm.sol=lm(y∼x,data=mydata)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�í�©Û:
y.fit=predict(lm.sol)e.hat=y-y.fit /*½öe.hat=residuals(lm.sol)*/e.std=rstandard(lm.sol)plot(e.hat∼y.fit)plot(e.std∼y.fit)plot(e.hat∼x)
ÊÏí�ã��±ÏLXe�ª��:
plot(lm.sol,which=1)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
lí�ã�wÑ, ù´��/i.í�ã, ´��à5Ø�ÎÜ���wG. �ÄéÏCþy�C�, }Áz =
√y, �£8�§
z = 0.5822 + 0.000953x.
R§S:
z=sqrt(y)mydata2=data.frame(z,y,x)lm.sol2=lm(z∼x,data=mydata2)summary(lm.sol2)z.fit=predict(lm.sol2)e.hat=z-z.fit /*½öe.hat=residuals(lm.sol2)*/plot(e.hat∼z.fit) /*½ö^plot(lm.sol2,which=1)*/
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
#�í�ãØ¥y?Û²w5K5, ùL²·�¤^�C�´Ü·�. ���£8�§�
y = z2 = (0.5822+0.000953x)2 = 0.339+0.0011x+0.00000091x2.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�e5·��êâ��ä: É~:�äÚrK�:�ä.
É~:�ä : duÆ)zí�ri�Cqw¤´�pÕá�ÑlN(0, 1), @o|ri| > 2´��Vǯ�, u)�VÇ��0.05. Ïd, ek,�|ri| > 2, ·�Òknd~¦éA���:(x
′i, yi)´
É~:.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
rK�:�ä : kÚ?�PÒ, ^Y(i), X(i)Úe(i)©OL«lY , XÚe¥GØ1i1¤����þ½Ý. GØ1i|êâ�,�e�n− 1|êâ��5£8�.�
Y(i) = X(i)β + e(i), E(e(i)) = 0, Cov(e(i)) = σ2In−1. (3.4.6)
rlù��.¦��β�LSEP�β(i), K
β(i) = (X′
(i)X(i))−1X
′
(i)Y(i). (3.4.7)
�þβ − β(i)�N1i|êâé£8Xê�O�K���, �§´���þ, ØBuA^©Û, A�ħ�,«êþz¼ê. Cookål´Ù¥A^�2���«.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Äk·�5¦β − β(i)�°(L�ª. �d, Ik0���ð�ª:
Ún
�A�n× n�_Ý, uÚvþ�n× 1�þ, @ok
(A− uv′)−1 = A−1 +
A−1uv′A−1
1− v′A−1u.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Px′i��OÝX�1i1. @oX = (x1, · · · ,xn)
′. |^þãÚ
n, ��
(X′
(i)X(i))−1 = (X
′X − xix
′i)−1
= (X′X)−1 +
(X′X)−1xix
′i(X
′X)−1
1− hii,(3.4.8)
Ù¥hii = x′i(X
′X)−1xi�lfÝH�1i�é����, �¡
�m\:. e,hii���, K�¡�pm\:. qÏ�
X′
(i)Y(i) =∑j 6=ixjyj =
n∑j=1
xjyj − xiyi = X′Y − xiyi,
¤±
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
β(i) = (X′
(i)X(i))−1X
′
(i)Y(i)
=[(X
′X)−1 +
(X′X)−1xix
′i(X
′X)−1
1− hii
](X
′Y − xiyi)
= β − (X′X)−1xiyi +
(X′X)−1xix
′iβ
1− hii
−(X′X)−1xihiiyi1− hii
= β − (X′X)−1xiyi1− hii
+(X
′X)−1xix
′iβ
1− hii
= β − (X′X)−1xiei1− hii
. (3.4.9)
¤±
β − β(i) =(X
′X)−1xiei1− hii
.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
CookÚ?e��ål:
Di(M , c) =(β(i) − β)
′M(β(i) − β)
c,
Ù¥M´�½��½Ý, c´�½��~ê. N´wÑ
Di(M , c) =e2i
c(1− hii)2· x′
i(X′X)−1M(X
′X)−1xi.
�M = X′X, c = (p+ 1)σ2, K
Di =e2i
(p+ 1)σ2(1− hii)2· hii =
1
p+ 1· hii
1− hii· r2i .
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.4.2)
Cookål
Di =(β(i) − β)
′X
′X(β(i) − β)
(p+ 1)σ2
=e2i
(p+ 1)σ2(1− hii)2· hii
=1
p+ 1· hii
1− hii· r2i , i = 1, · · · , n, (3.4.10)
Ù¥hii�lfÝH�1i�é��, ri´Æ)zí�.
ù�½nL², 3O�Cookål��ÿ, ·��I�l��êâ��5£8�.�ÑÆ)zí�ri ÚlfÝ�é��hiiÒ�±, Ø7é?Û��Ø��êâ��5£8�.(3.4.6) ?1O�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
hii�¹Â: hiiÝþ1i|êâxi�¥%x = 1n
∑ni=1 xi�ål.
��.
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In,
b�gCþ®¥%z, K
X =
1 x11 − x1 · · · x1p − xp...
......
...1 xn1 − x1 · · · xnp − xp
=:
1 (x1 − x)′
......
1 (xn − x)′
,
ùpx = (x1, · · · , xp)′, xi = (xi1, · · · , xip)
′. ²{üO���
X′X =
(n 00 L
),
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ù¥L =∑n
i=1(xi − x)(xi − x)′, �p��. ?�Ú��
hii =1
n+ (xi − x)
′L−1(xi − x).
e´��¥%z�5£8�., K
X =
1 x1 − x...
...1 xn − x
, X′X =
(n 00 Sxx
), hii =
1
n+
(xi − x)2
Sxx.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3úª(3.4.10)¥, 1/(p+ 1)�iÃ', Pi = hii1−hii´hii�üNO¼
ê, Ï�hiiÝþ1i|êâxi�¥%x = 1n
∑ni=1 xi�ål. Ï
dPi�x1i|êâålÙ§êâ��C. Cookål�PiÚr2i�
��¤û½. ½n3.4.2w�·�, pm\:�U´rK�:, ��UØ´; É~:�U´rK�:, ��UØ´; �XJ�|êâQ´pm\:q´É~:, @o§Ò´rK�:.
��Cookål��^±�½rK�:��.�´é(J�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A^�&ý¥�±éCookålí�¥�M , c�À����½�nØ|±. 3Ø���b�e,
β ∼ N(β, σ2(X′X)−1),
|^íØ2.4.1�
(β − β)′X
′X(β − β)
σ2∼ χ2(p+ 1),
,��¡(n− p− 1)σ2
σ2∼ χ2(n− p− 1),
��ö�pÕá. ¤±
(β − β)′X
′X(β − β)
(p+ 1)σ2∼ F (p+ 1, n− p− 1). (3.4.11)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
8Ü
S ={β :
(β − β)′X
′X(β − β)
(p+ 1)σ2≤ Fα(p+ 1, n− p− 1)
}¡�β��&Y²�1− α��&ý¥. (3.4.10)�(3.4.11)¥�ÚOþé�q, �cö¿ØÑlF©Ù. , /Ïu�ö·��±éDi�����ѱe�VÇ)º.
~X, eDi = F0.50(p+ 1, n− p− 1), KL²1i|êâ(x′i, yi)�
GØ�, β��Oβ(i)£Ä�β��&Y²�0.5��&ý¥>.
þ; eDj = F0.80(p+ 1, n− p− 1), Kβ(j)£Ä�β��&Y²�0.2��&ý¥>.þ. ��1i|êâé�O�K�'1j|êâ5��.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éu~3.4.1, ·��Xe�êâ�ä©Û:
R§S:
yx=read.table(”ex p47 data.txt”)x=yx[,1]y=yx[,2]z=sqrt(y)mydata2=data.frame(z,x)lm.sol2=lm(z∼x,data=mydata2)summary(lm.sol2)z.fit=predict(lm.sol2)r.std=rstandard(lm.sol2)plot(r.std∼z.fit)text(z.fit,r.std,type=”1:53”)r.stdinfluence.measures(lm.sol2)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
©Û: 126Ò��:�É~:. ek”�cü�”�:, K�ÑÆ)zí
���?1?�Ú(@.Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
o(: 126Ò��:�É~:, ·�I�u�êâ5 ´ÄkL�, eÃL�KíØd��:. d, 126Ò��:�rK�:,IÚå5¿.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.4.2�åÿÁêâ. L3.4.2´��Æ[ÿÁ�21��Ö�P¹, Ù¥x´�Ö�c#(ü : �), y´,«�å�I, ÏLùêâ, ·��ïá�å�c#Cz�'X.
L3.4.2�åÿÁêâ
SÒ x y SÒ x y1 15 95 12 9 962 26 71 13 10 833 10 83 14 11 844 9 91 15 11 1025 15 102 16 10 1006 20 87 17 12 1057 18 93 18 42 578 11 100 19 17 1219 8 104 20 11 86
10 20 94 21 10 10011 7 113
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R §S:
x=c(15,26,10,9,15,20,18,11,8,20,7,9,10,11,11,10,12,42,17,11,10)y=c(95,71,83,91,102,87,93,100,104,94,113,96,83,84,102,100,
105,57,121,86,100)lm.sol=lm(y∼x)summary(lm.sol)influence.measures(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
plot(lm.sol,which=2) /*QQã*/y.fit=predict(lm.sol)e=y-y.fitr.std=rstandard(lm.sol)plot(r.std∼y.fit,xlab=expression(hat(y)),ylab=“r”)text(y.fit,r.std,type=“1:25”)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
o(: 119Ò��:�É~:, 118,19Ò���:´rK�:.dQQãÚí�ã, ·��É�5b�!��à5b�!Ø�'5b�Ú��5b�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC�
éu*ÿêâ, e²L£8�ä���, §�Ø÷v�5b�!��à5b�!Ø�'5b�Ú��5b�¥���½eZ�,@o·�Ò�éêâæ�”£�”��. ¢�y², êâC�´?nk¯Kêâ��«Ð�{.
�!0�Box-CoxC�, §�Ì�A:´Ú\��ëê, ÏLêâ���OTëê, l (½Aæ��êâC�/ª. ¢�y²,Box-CoxC�éNõ¢SêâÑ´1�k��.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC�´éÏCþ�XeC�:
y(λ) =
{yλ−1λ , λ 6= 0,
ln y, λ = 0,(3.5.1)
ùpλ´���½�C�ëê.
Box-CoxC�´�xC�, §�)Nõ~��C�, ÃXéêC�(λ = 0), �êC�(λ = −1)Ú²��C�(λ = 1/2)��.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éÏCþ�n�*ÿ�y1, · · · , ynA^þãC�, ��C����þ
Y (λ) = (y(λ)1 , · · · , y(λ)
n )′.
·��(½C�ëêλ¦�Y (λ)÷v
Y (λ) = Xβ + e, e ∼ N(0, σ2I). (3.5.2)
=�¦C����þY (λ)�£8gCþ�mäk�5�''X,Ø�÷v��©Ù!��à5!�pÕá.
Ïd, Box-CoxC�´ÏLéëêλ�ÀJ, ��é�5êâ�”nÜ£n”, ¦Ù÷v�����5£8�.�¤kb�^�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�^4�q,�{5(½λ. é�½�λ, βÚσ2, Y (λ)�q,¼ê�
1
(√
2πσ)nexp
{− 1
2σ2(Y (λ) −Xβ)
′(Y (λ) −Xβ)
},
¤±Y�q,¼ê�
L(β, σ2) =1
(√
2πσ)nexp
{− 1
2σ2(Y (λ) −Xβ)
′(Y (λ) −Xβ)
}|J |,
ùpJ�C��Jacobi1�ª
J =
n∏i=1
dy(λ)i
dyi=
n∏i=1
yλ−1i .
lnL(β, σ2)'uβÚσ2¦�¿-Ù�u", ��βÚσ2�MLE�{β(λ) = (X
′X)−1X
′Y (λ),
σ2(λ) = 1nY
(λ)′(I −H)Y (λ) ∆
= 1nRSS(λ,Y (λ)).
(3.5.3)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éA�q,¼ê����
Lmax(λ) = L(β(λ), σ2(λ))
= (2πe)−n/2 · |J | ·(RSS(λ,Y (λ))
n
)−n/2.(3.5.4)
ù´λ�¼ê,·�ÏL¦§����5(½λ. ù�duÏL¦lnLmax(λ)����5(½λ. Äk, k
lnLmax(λ) = −n2· ln[RSS(λ,Y (λ))
]+ ln |J |+ C
= −n2
ln[Y (λ)
′
|J |1/n(I −H)
Y (λ)
|J |1/n]
+ C
∆= −n
2ln[RSS(λ,Z(λ))
]+ C, (3.5.5)
Ù¥Z(λ) = (z(λ)1 , · · · , z(λ)
n )′
= Y (λ)/|J |1/n,
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
RSS(λ,Z(λ)) = Z(λ)′(I −H)Z(λ). (3.5.6)
(3.5.5)ªéBox-CoxC�3O�Åþ�¢y�5é��B, Ï��¦lnLmax(λ)����, �I¦RSS(λ,Z(λ))����. �,éJé�¦RSS(λ,Z(λ))������λ�)ÛL�ª, �é�X��½�λ�, ÏL¦���¦�O�£8§S, N´O�ÑéA�RSS(λ,Z(λ)). ,�xÑRSS(λ,Z(λ))'uλ�ã, lãþ�±Cq�éѦRSS(λ,Z(λ)) ������λ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Box-CoxC��äNÚ½:
1. é�½�λ�,O�z(λ)i , i = 1 · · · , n.
2. U(3.5.6)ªO�í�²�ÚRSS(λ,Z(λ)).
3. é�X��½�λ�, EþãÚ½, ���A�í�²�ÚRSS(λ,Z(λ))��G�. ±λ�î¶, RSS(λ,Z(λ)) �p¶, xÑ�A��. ^�*�{, éѦRSS(λ,Z(λ))������λ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.5.13~3.4.1¥, ·�éÏCþy�²��C�, ù��u¦^λ = 0.5�Box-CoxC�. ·�y35y¢ù��C�´Ü·�.eL�Ñ12�ØÓ�λ�éA�í�²�ÚRSS(λ,Z(λ)), {ü'���uy�λ = 0.5�í�²�ÚRSS(λ,Z(λ))����. ÏdCq/@�0.5Ò´C�ëêλ��`ÀJ.
L3.5.1
λ -2 -1 -0.5 0 0.125 0.25RSS 34101.04 986.04 291.59 134.10 119.20 107.21λ 0.375 0.5 0.625 0.75 1 2
RSS 100.26 96.95 97.29 101.69 127.87 1275.56
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
x<-c(679,292,1012,493,582,1156,997,2189,1097,2078,1818,1700,747,2030,1643,414,354,1276,745,435,540,874,1543,1029,710,1434,837,1748,1381,1428,1255,1777,370,2316,1130,463,770,724,808,790,783,406,1242,658,1746,468,1114,413,1787,3560,1495,2221,1526)y<-c(0.79,0.44,0.56,0.79,2.70,3.64,4.73,9.50,5.34,6.85,5.84,5.21,3.25,4.43,3.16,0.50,0.17,1.88,0.77,1.39,0.56,1.56,5.28,0.64,4.00,0.31,4.20,4.88,3.48,7.58,2.63,4.99,0.59,8.19,4.79,0.51,1.74,4.10,3.94,0.96,3.29,0.44,3.24,2.14,5.71,0.64,1.90,0.51,8.33,14.94,5.11,3.85,3.93)lm.sol<-lm(y∼x)library(MASS)boxcox(lm.sol,plotit=T,lambda=seq(-2,2,by=0.05))
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
2Â���¦�O
3c¡�?Ø¥, ·�o´b��5£8�.�Ø�´��à5�Ø�'�, =Cov(e) = σ2In.�´3Nõ¢S¯K¥, êâ Ø÷vù�b�(�ÏLí�ã�ä). ù�·�Ib�Ø��þ����Ý�Cov(e) = σ2Σ, ùpΣ´���½Ý. ù�Σ�¹��ëê, �·�ùpb�Σ ´��®��.
·��?Ø��5£8�.�
Y = Xβ + e, E(e) = 0, Cov(e) = σ2Σ. (3.6.1)
Ì�8�´�Oβ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Σ´é¡�½Ý, ¤±�3n× n���P¦�
Σ = PΛP′,
ùpΛ = diag(λ1, · · · , λn), λi > 0, i = 1, · · · , n, ´Σ�A��.P
Σ12 = P diag(λ
121 , · · · , λ
12n )P
′,
K��(Σ12 )2 = Σ, ¡Σ
12´Σ�²��. Σ−
12L«Σ
12�_Ý
.
·�r�5£8�.(3.6.1)?1��C�. ^Σ−12�¦(3.6.1). P
Z = Σ−12Y , U = Σ−
12X, ε = Σ−
12e.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�Cov(ε) = Σ−12σ2ΣΣ−
12 = σ2In, u´·��Xe��5£
8�.
Z = Uβ + ε, E(ε) = 0, Cov(ε) = σ2In. (3.6.2)
ù´·�®?ØL��.. 3ù#�.¥, ��β�LSE�
β∗ = (U′U)−1U
′Z = (X
′Σ−1X)−1X
′Σ−1Y . (3.6.3)
·�¡β∗�β�2Â���¦�O(GLSE). ù��OäkûÐ�ÚO5�.
½n (3.6.1)
(a) E(β∗) = β;(b) Cov(β∗) = σ2(X
′Σ−1X)−1;
(c)é?¿�p+ 1���þc, c′β∗�c
′β��������5Ã
�O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
y²: (a)
E(β∗) = (X′Σ−1X)−1X
′Σ−1E(Y ) = (X
′Σ−1X)−1X
′Σ−1Xβ = β.
(b)|^½n2.1.3,
Cov(β∗) = Cov[(X′Σ−1X)−1X
′Σ−1Y ]
= (X′Σ−1X)−1X
′Σ−1Cov(Y )((X
′Σ−1X)−1X
′Σ−1)
′
= σ2(X′Σ−1X)−1X
′Σ−1Σ((X
′Σ−1X)−1X
′Σ−1)
′
= σ2(X′Σ−1X)−1.
(c)�b′Y´c
′β�?��5à �O. éu�.(3.6.2),
c′β∗ = c
′(U
′U)−1U
′Z, b
′Y = b
′Σ
12 Σ−
12Y = b
′Σ
12Z,
=c′β∗�c
′β�LSE, b
′Y = b
′Σ
12Z�c
′β��5à �O. ¤±
é�.(3.6.2)A^Gauss-Markov½n�
Var(c′β∗) ≤ Var(b
′Σ
12Z) = Var(b
′Y ),
�Ò¤á��=�c′β∗ = b
′Σ
12Z = b
′Y .
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n3.6.1(c)Ò´���/e�Gauss-Markov½n, §L²3���5£8�.(3.6.1)¥, GLSE β∗´�`�(eΣ = In, KGLSEòz�LSE β). éu�.(3.6.1), N´y²βE´Ã �O, ��7´�`��5à �O, Ï�Var(c
′β∗) ≤ Var(c
′β). ùÒ´`,
éu���5£8�.(3.6.1), GLSEo´`uLSE.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�.(3.6.1)�{ü�~f´ÏCþ�ØÓ*ÿäkØ�����/, =
Cov(e) = diag(σ21, · · · , σ2
n),
ùp�σ2i , i = 1, · · · , nØ���. Px
′1, · · · ,x
′n©O´�OÝ
X�n�1�þ. N´íÑ,
β∗ =( n∑i=1
xix′i
σ2i
)−1( n∑i=1
xiyiσ2i
). (3.6.4)
ü�Úª©O´xix′iÚxiyi�\�Ú(�Ñ�1/σ2
i ), Ïd�¡β∗�\����¦�O(WLSE).
σ2i ´���, ù�·�I��{¦�§���Oσ2
i , ,�3(3.6.4)¥^σ2
i�Oσ2i . ù«�O�{¡�üÚ�O(two-stage
estimate).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.6.1b�·�^�«°�¤ì3ü�¢�¿éÓ��þµ©O?1n1gÚn2gÿþ, Pùÿþ�©O�y11, · · · , y1n1Úy21, · · · , y2n2 . r§��¤�5£8�./ª{
y1i = µ+ e1i, i = 1, · · · , n1,y2i = µ+ e2i, i = 1, · · · , n2.
duü�¢�¿��*^�9¤ì�°ÝØÓ, �§��ÿþØ����Ø�. �
Var(e1i) = σ21, Var(e2i) = σ2
2, σ21 6= σ2
2.
Pe = (e11, · · · , e1n1 , e21, · · · , e2n2)′, K
Cov(e) =
(σ2
1In1 00 σ2
2In2
)= σ2
2
(θIn1 0
0 In2
)∆= σ2
2Σ,
ùpθ = σ21/σ
22. b��, KӮ�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿�ùp��OÝX = (1, · · · , 1)′, u´µ�GLSE�
µ∗ =(n1
θ+ n2
)−1(1
θ
n1∑i=1
y1i +
n2∑i=1
y2i
).
P
y1 =1
n1
n1∑i=1
y1i, y2 =1
n2
n2∑i=1
y2i,
ω1 =1
Var(y1)=n1
σ21
, ω2 =1
Var(y2)=n2
σ22
.
Kµ∗�U��
µ∗ =ω1
ω1 + ω2y1 +
ω2
ω1 + ω2y2.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
=µ∗´ü�¢�¿*ÿ�þ��\�²þ, §��� ω1ω1+ω2
Úω2
ω1+ω2��¢�¿ÿþ�Ø���Úÿþgêk'. Ø����
�, ÿþgê��, éA��Ò�.
µ∗�¹��ëêσ21Úσ
22, ØUGâSA^. ·��±�{�
Eσ21Úσ
22��O. ¯¢þ, ùü�¢�¿�*ÿêâ©O�¤�
5£8�.Yi = µ1ni + ei, i = 1, 2,
ùpYi = (y1i, · · · , y1ni)′, ei = (ei1, · · · , eini)
′. Ï�Cov(ei) =
σ2i Ini , ¤±ei, i = 1, 2÷vGauss-Markovb�. ¤±σ2
i�LSE�
σ2i =
1
ni − 1‖Yi − yi1ni‖2 i = 1, 2.
^σ2i , i = 1, 2�Oµ∗¥�σ2
i , i = 1, 2, =���µ�üÚ�O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
õ��5
£8Xê�LSEkNõ`û�5�, Ù¥���´Gauss-Markov½n, §L²3�5à �Oa¥, LSE´���äk������O. �´ù�`:, ¦�LSE3�5ÚO�.��OnØÚ¢SA^¥Ókýé��/ .
�´·�±c?Ø�LSEI�b��OÝX´�÷��, =�¦ÝX���þ�m´�5Ã'�. , , 3¢S¯K¥, du²~�?n¹k�õgCþ��.£8¯K, �²LCþ�m Ø´�á� ´�péX�, ùÑ�¦�OÝX���þ�mØ�U���5Ã'. éõ�¹e, �OÝX���þ�m�3õ��5/E��5(multi-collinearity)'X.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
e�3Ø��0�p+ 1�~êc0, c1, · · · , cp¦�
c0 + c1xi1 + · · ·+ cpxip = 0, i = 1, · · · , n,
K¡gCþx1, · · · , xp�m�3X���õ��5/E��5'X.
3¢S¯K¥, ���õ��5/E��5'X¿Øõ�, ��Ñy�´�½§Ýþ���5.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â (õ��5/E��5'X)
e�3Ø��0�p+ 1�~êc0, c1, · · · , cp¦�
c0 + c1xi1 + · · ·+ cpxip ≈ 0, i = 1, · · · , n,
K¡gCþx1, · · · , xp�m�3Xõ��5/E��5'X.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
é²Lêâï��, õ��5��/éõ, õ��5�/¬�õ��5£8©Û�5�oK�!XÛ�ägCþ�m�õ��5±9XÛ�Ñõ��5�K��¯Kò´�!�?Ø�Ì�SN.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�kÚ\��Vg: þ�Ø�(MSE: Mean Squared Errors), §´^5µd���O`��IO�.
½Â
�θ����þ. θ�θ����O. ½Âθ�þ�Ø��
MSE(θ) = E‖θ − θ‖2 = E[(θ − θ)′(θ − θ)].
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.7.1)
MSE(θ) = tr[Cov(θ)] + ‖Eθ − θ‖2.
y²: ØJwÑ
MSE(θ) = E[(θ − θ)′(θ − θ)]
= E[(θ − Eθ) + (Eθ − θ)]′[(θ − Eθ) + (Eθ − θ)]
= E(θ − Eθ)′(θ − Eθ) + E(Eθ − θ)
′(Eθ − θ)
∆= ∆1 + ∆2.
|^,�5�,
∆1 = E{tr[(θ − Eθ)′(θ − Eθ)]}
= E{tr[(θ − Eθ)(θ − Eθ)′]}
= tr[E(θ − Eθ)(θ − Eθ)′] = tr[Cov(θ)].
∆2 = E[(Eθ − θ)′(Eθ − θ)] = ‖Eθ − θ‖2´w,�. y..
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ePθ = (θ1, · · · , θp+1)′, K
∆1 =
p+1∑i=1
Var(θi),
§´θ�©þ����Ú.
∆2 =
p+1∑i=1
(Eθi − θi)2,
§´θ�©þ� ²��Ú. ¤±, ���O�þ�Ø�d§���Ú �¤û½. ��Ð��OAk�����Ú �.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n
3�5£8�.(3.1.5)¥, éβ�LSE β, k(a) MSE(β) = σ2
∑p+1i=1
1λi;
(b) E‖β‖2 = ‖β‖2 + σ2∑p+1
i=11λi,
Ù¥λ1, · · · , λp+1 > 0�X′X�A��.
y²: (a)Ï�LSE β´Ã �O, ¤±∆2 = 0,
MSE(β) = ∆1 = tr[Cov(β)] = σ2tr[(X′X)−1].
Ï�X′X´é¡�½Ý, ¤±�3��P¦�
X′X = P diag(λ1, · · · , λp+1)P
′,
ùpλ1, · · · , λp+1 > 0�X′X�A��. ¤±
(X′X)−1 = P diag(
1
λ1, · · · , 1
λp+1)P
′.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
|^,�5�êþ��
tr(X′X)−1 = tr
(diag(
1
λ1, · · · , 1
λp+1))
=
p+1∑i=1
1
λi.
¤±MSE(β) = σ2∑p+1
i=11λi
.
(b) �
MSE(β) = E[(β − β)′(β − β)]
= E(β′β − 2β
′β + β
′β)
= E‖β‖2 − β′β,
u´
E‖β‖2 = ‖β‖2 + MSE(β) = ‖β‖2 + σ2p+1∑i=1
1
λi.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(Ø(a)w�·�, XJX′X��k��A���~�, =�~�
Cu", @oMSE(β)Ò¬é�. lþ�Ø��IO5w, ���¦�OβØ´��Ð��O. ùÚGauss-Markov½n¿Øgñ, Ï�Gauss-Markov½n==�y���¦�O3�5à �Oa¥�����5. �3X
′X��k��A���~��, ù��
�������%é�, Ï ��é��þ�Ø�.
(Ø(b)w�·�, XJX′X��k��A���~�, @o��
�¦�Oβ��ݲþ`5�'ý��β��Ý�éõ. ùÒ��β�,©þ�ýé�L�.
o�, �X′X��k��A���~��, ���¦�OØ2´
����O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
·�5©Û”��k��A���~�”3�OÝX½ö£8gCþþ¿�X�o.
PX = (1,x1, · · · ,xp), =xi�X�1i+ 1�. �λ�X′X���
A��, φ�ÙéA�A��þ, Ù�Ý�1, =φ′φ = 1.
eλ ≈ 0, K
‖Xφ‖2 = φ′X
′Xφ = λφ
′φ = λ ≈ 0.
u´Xφ ≈ 0. Pφ = (c0, c1, · · · , cp)′, K
c0 + c1x1 + · · ·+ cpxp ≈ 0. (3.7.1)
=�OÝX���þ�m(=gCþ�m)kõ��5'X.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��, e�OÝX���þ�mkõ��5'X, =(3.7.1)¤á, d�X
′XE´�½Ý�|X ′
X| ≈ 0. dd��
p+1∏i=1
λi = |X ′X| ≈ 0,
¤±��k��A���~�, �Cu".
�Ò´`��k��A���~��X���þ�mkõ��5'X´�d�. ù�¡�OÝX´¾�Ý.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
õ��5��ä :
(1) ��)äÏf(VIF: variance inflation factor)�ä{
PR2j�gCþxjéÙ{p− 1�gCþ��½Xê, ½Â
VIFj =1
1−R2j
, j = 1, · · · , p.
duR2jÝþgCþxjéÙ{p− 1�gCþ�m��5�'§
Ý, x1, · · · , xp�m�õ��5�î, R2j��Cu1, VIFj�Ò
��. Ïd^VIF 5Ýþõ��5�§Ý´Ün�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ýþ�OK:
�k,�VIFj ≥ 10
�, @�gCþ�m�3î�õ��5; ½ö�
VIF =1
p
p∑i=1
VIFj � 1
�, @�gCþ�m�3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(2)A���^�ê(CI: Condition Index)�ä{
��Øþj�K�, ·�b�gCþ�ÏCþ�*ÿ�þ®IOz. d��@��5£8�.vk�å�, �OÝX´n× p�Ý, X
′X´p�gCþ����'XêÝ.
A���ä{: XJX′Xkm�A��Cq�", @oXÒkm
�õ��5'X, ¿�ùm�õ��5'X�Xê�þÒ´ùm��Cu"�A��¤éA�IO��zA��þ.
^�ê�ä{: b�X′X�p�A��©O�λ1, · · · , λp, Ù¥�
�A���λmax, ��A���λmin, ¡
κj =λmaxλj
, j = 1, · · · , p
�A��λj�^�ê.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
P
κ = maxjκj =
λmaxλmin
.
§�±^5ÝþÝX′X�A��Cq�"�§Ý, Ïd�±^
5�äõ��5´Ä�3±9õ��5�î§Ý.
^��OK:
e0 < κ < 100, K@�Ø�3õ��5;
e100 ≤ κ ≤ 1000, K@��3�r�õ��5;
eκ > 1000, K@��3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.7.1.�Ä��k8�£8gCþ��5£8¯K, �©êâ�eL.
L3.7.1
SÒ y x1 x2 x3 x4 x5 x61 10.006 8 1 1 1 0.541 -0.0992 9.737 8 1 1 0 0.130 0.0703 15.087 8 1 1 0 2.116 0.1154 8.422 0 0 9 1 -2.397 0.2525 8.625 0 0 9 1 -0.046 0.0176 16.289 0 0 9 1 0.365 1.5047 5.958 2 7 0 1 1.996 -0.8658 9.313 2 7 0 1 0.228 -0.0559 12.960 2 7 0 1 1.380 0.502
10 5.541 0 0 0 10 -0.798 -0.39911 8.756 0 0 0 10 0.257 0.10112 10.937 0 0 0 10 0.440 0.432
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
yx=read.table(”ex p62 data.txt”)y=yx[, 1]x1=yx[, 2]x2=yx[, 3]x3=yx[, 4]x4=yx[, 5]x5=yx[, 6]x6=yx[, 7]mydata=data.frame(y,x1,x2,x3,x4,x5,x6)lm.sol=lm(y∼ x1+x2+x3+x4+x5+x6,data=mydata)summary(lm.sol)library(DAAG)vif(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
VIFÑÑ(J
x1 x2 x3 x4 x5 x6
182.0500 161.3600 266.2600 297.7100 1.9200 1.4553
Ï�co�VIPÑØ�u10, ¤±@�gCþ�m�3î�õ��5.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A���^�ê�ä{:
R§S:
X=cbind(x1,x2,x3,x4,x5,x6)Xrho=cor(X)rhoeigen(rho)kappa(rho,exact=TRUE) /*%@´exact=FALSE, ù�k���O�Ø�*/
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
A��!A��þ�^�ê:
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
λmin = 0.001106051, éA�A��þ�
(0.44768, 0.42114, 0.541689, 0.57337, 0.00605, 0.00217)′.
¤±IOzgCþx∗i , i = 1, · · · , 6�m�3Xe�õ��5'X
0.44768x∗1 + 0.42114x∗2 + 0.541689x∗3 + 0.57337x∗4 + 0.00605x∗5
+0.00217x∗6 ≈ 0.
ù`²éu�©gCþxi, i = 1, · · · , 6, �3~êc0¦�
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4
+0.00605x5 + 0.00217x6 ≈ 0.
½��
c0 + 0.44768x1 + 0.42114x2 + 0.541689x3 + 0.57337x4 ≈ 0.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
^�êκ = 2195.908 > 1000, �`²gCþ�m�3î�õ��5'X.
�Øõ��5��{:
O���Nþ, �ؽ�)gCþ��5�'5.
ã+à 5, Ïék �O, ·�ò0�*�OṲ́©�O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O
�gCþ�mäkõ��5�, ��ÑLSE²wC��¯K,Hoerlu1962cJÑ�«U?����¦�O�{, =*�O(Ridge Estimate). ��, HoerlÚKennard31970céT�O�?�Ú��[?Ø.
*�O�g�: �gCþ�m�3õ��5�, �OÝX´¾��, =|X ′
X| ≈ 0, l (X′X)−1�CÛÉ. �;�ù�y�,
�X′X \þ���~êé�ÝkI(k > 0),KÝ
(X′X + kI)−1
�CÛÉ��U5�'(X′X)−1�CÛÉ��U5��õ. Ïd
^
β(k) = (X′X + kI)−1X
′Y (3.8.1)
����ëêβ��OAT'���¦�O�½�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â
é�½�0 < k <∞, ¡β(k) = (X′X + kI)−1X
′Y�£8X
êβ�*�O. d*�O¤ïá�£8�§¡�*£8�§.¡k�*ëê. éuβ(k)�©þβj(k), r3²¡���IX
¥βj(k)�kCz¤LyÑ5��¡�*,(ridge trace).
5: kØÓ, ·���ØÓ��O. Ïd*�Oβ(k)´���Oa.�k = 0�, β(k)Ò´Ï~�LSE. ���¹e, ·�Jå*�O,´Ø�)LSE�.
3?1*�O�c, ��Øþj�K�, ·�ob�gCþ�ÏCþþ®IOz, Ïdùp��OÝX´n× pÝ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O�5� :
5�1: β(k)´β�k �O, =é?¿�0 < k <∞, E(β(k)) 6=β.
k 5´*�O����¦�O�����ØÓ�?. ���O�þ�Ø�d���ÚÚ ��²�Ú|¤. ��3õ��5�, ���¦�O�,�± �Ü©�", �§���Ü©%é�, �ª��§�þ�Ø�é�. ·�Ú?*�O�8�´ã+à 5, ����Ü©��ÌÝ~�, �ªü$Ùþ�Ø�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�2: β(k)´���¦�Oβ����5C�.
y²: �I5¿�
β(k) = (X′X + kI)−1X
′Y
= (X′X + kI)−1X
′X · (X ′
X)−1X′Y
= (X′X + kI)−1X
′Xβ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�3: é?¿�k > 0, e‖β‖ 6= 0, K·�ok‖β(k)‖ < ‖β‖.=*�O´r���¦�Oβ��:�·Ý�Ø ���, *�O´��Ø k �O.
y²: �Äõ��5£8�.Y = Xβ + e, -
Z = XP , α = P′β,
Ù¥P���Ý÷v
P′(X
′X)P = Λ = diag(λ1, · · · , λp),
λ1, λ2, · · · , λp > 0�X′X�A��. ù�, õ��5£8�.�
��
Y = Zα+ e, E(e) = 0, Cov(e) = σ2In. (3.8.2)
·�¡(3.8.2)��5£8�.�;K/ª, ¡α�;K£8Xê.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿�Z′Z = P
′X
′XP = Λ, ¤±
α = (Z′Z)−1Z
′Y = Λ−1Z
′Y .
β = (X′X)−1X
′Y = PΛ−1P
′X
′Y = PΛ−1Z
′Y = Pα.
§��A�*�O©O�
α(k) = (Z′Z + kI)−1Z
′Y = (Λ + kI)−1Z
′Y ,
β(k) = (X′X + kI)−1X
′Y
= PP′(X
′X + kI)−1PP
′X
′Y
= Pα(k).
Ïd
‖β(k)‖ = ‖α(k)‖ = ‖(Λ + kI)−1Λα‖ < ‖α‖ = ‖β‖.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5: ´�þ�Ø�3�OÚëê���C�e�±ØC, ¤±;K£8Xê����¦�O(½*�O)Ú�£8Xê����¦�O(½*�O)k�Ó�þ�Ø�:
MSE(α) = MSE(β), MSE(α(k)) = MSE(β(k)). (3.8.3)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.8.1, *�O�35½n)
�3k > 0¦�MSE(β(k)) < MSE(β).
=�3k > 0, ¦�3þ�Ø�¿Âe, *�O`u���¦�O.
y²: d(3.8.3), �Iy²�3�3k > 0¦�
MSE(α(k)) < MSE(α). (3.8.4)
Pf(k) = MSE(α(k)), k ≥ 0. 5¿f(0) = MSE(α). e·�Uy²f(k)3[0,∞)þ´ëY¼ê�f
′(0) < 0, K7�3�����
k > 0 ¦�(3.8.4)¤á.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5?Øf(k). 5¿�
E(α(k)) = (Λ + kI)−1Z′E(Y )
= (Λ + kI)−1Z′Zα
= (Λ + kI)−1Λα,
�
Cov(α(k)) = σ2(Λ + kI)−1Z′Z(Λ + kI)−1
= σ2(Λ + kI)−1Λ(Λ + kI)−1.
¤±
f(k) = MSE(α(k)) = tr[Cov(α(k))
]+ ‖E(α(k))−α‖2
= σ2p∑i=1
λi(λi + k)2
+ k2p∑i=1
α2i
(λi + k)2
∆= f1(k) + f2(k). (3.8.5)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
w,f(k)´[0,∞)þ�ëY¼ê. q
f′1(k) = −2σ2
p∑i=1
λi(λi + k)3
, f′1(0) = −2σ2
p∑i=1
1
λ2i
< 0 (3.8.6)
±9
f′2(k) = 2k
p∑i=1
λiα2i
(λi + k)3, f
′2(0) = 0, (3.8.7)
¤±f′(0) = f
′1(0) + f
′2(0) = −2σ2
∑pi=1
1λ2i< 0. y..
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�O�35½n3nØþy²�3,�*�O`u���¦�O, ��éÑù�*ëêk´ØN´�. N´wÑ, nØþ, ù�k´e��§�):
f′(k) = −2σ2
p∑i=1
λi(λi + k)3
+ 2k
p∑i=1
λiα2i
(λi + k)3
= 2
p∑i=1
λi(kα2i − σ2)
(λi + k)3= 0. (3.8.8)
ù�)�6u��ëêαi, i = 1, · · · , pÚσ2, ¤±Ø�Ul)�§��ݼ�*ëêk. ÚOÆ[�lÙ§å»JÑÀJ*ëêk��{.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*ëêÀJ�{ :
(1) Hoerl-Kennardúª
HoerlÚKennardJÑ�ÀJk�úª´
k =σ2
maxiα2i
. (3.8.9)
¼�ù�úª��{Xe: d(3.8.8)�, ekα2i − σ2 < 0éi = 1,
· · · , pѤá, Kf′(k) < 0. u´�
k∗ =σ2
maxiα2i
,
�0 < k < k∗�, f′(k)o´�u", Ï f(k)3(0, k∗)þ´üN4
~¼ê, 2df(k)3[0,∞)�ëY5�f(k∗) < f(0). ,�2^αiÚσ2 �Oαi Úσ
2, B�(3.8.9).
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
(2) *,{
òβ1(k), · · · , βp(k)�*,x3�Üãþ, �â*,�Czª³ÀJk. ±e´A^ÀJk�OK:
�£8Xê�*�O��'�½;
^���¦�O�ÎÒØÜn�£8Xê, Ù*�O�ÎÒòC�Ün;
£8XêvkØÜn�ÎÒ;
í�²�ÚØ�þ,�õ.
���¹e, ·�ÀJ���k�U¦��^*,Ñm©ªu½.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.8.1{I²Ló�öF"ÏLISo��x1,�;þx2,o�¤þx3�ýÿ?�o�y, ±þCþ�ü þ��·{J. �dÂ81949-1959�11c�êâ, �eL.
c° x1 x2 x3 y1949 149.3 4.2 108.1 15.91950 161.2 4.1 114.8 16.41951 171.5 3.1 123.2 19.01952 175.5 3.1 126.9 19.11953 180.8 1.1 132.1 18.81954 190.7 2.2 137.7 20.41955 202.1 2.1 146.0 22.71956 212.4 5.6 154.1 26.51957 226.1 5.0 162.3 28.11958 231.9 5.1 164.3 27.61959 239.0 0.7 167.6 26.3
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
R§S:
library(MASS)yx=read.table(”ex p68 data.txt”)x1=yx[, 1]x2=yx[, 2]x3=yx[, 3]y=yx[, 4]mean(x1);mean(x2);mean(x3);mean(y)sd(x1);sd(x2);sd(x3);sd(y)mydata=data.frame(y, x1, x2, x3)lm.sol=lm(y∼x1+x2+x3)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
¼��Cþ���þ�!��IO�êâ:
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
¦^���¦�{�(J:
£8�§: y = −10.128− 0.051x1 + 0.587x2 + 0.287x3.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
£8©Û�(JL²: x1�£8Xê��O´Kê, ùØÎÜÙ²L¿Â, ù´Ï�{I´���á�?�I, �ISo��x1
O\�, ?�o�y�AO\. ¤±, £8Xê�ÎÒ�¢SØÎ.Ù�Ï´n�gCþ�m�3Xõ��5, ù�{ü/lx1,x2, x3����'XêÝwÑ:
Ïdx1�x3�3pÝ��'5.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�õ��5�ä, �±²(uygCþ�m�3Xõ��5:
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
^*�O�{Ïé*£8�§:
kéêâ?1IOz, R§S:
yxs=scale(yx)x1=yxs[, 1]x2=yxs[, 2]x3=yxs[, 3]y=yxs[, 4]mydata2=data.frame(x1,x2,x3,y)mydata2
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
rr.sol=lm.ridge(y∼0+x1+x2+x3,data=mydata2,lambda=c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))rr.sol /∗w«*�O∗/plot(rr.sol) /∗�*,ã∗/matplot(rr.sol$lambda,t(rr.sol$coef),type=”l”,col=c(”red”,”blue”,
”black”),main=”ridge trace”,xlab=expression(lambda),ylab=expression(hat(beta)(lambda))) /∗�*,ã∗/
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
*�Oêâ:
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�â*,ãÀJ*ëêk = 0.4, IOzCþ�*£8�§�
v = 0.416u1 + 0.213u2 + 0.531u3.
��, I�=�¤�©Cþ�*£8�§:
y − 21.891
4.544= 0.416× x1 − 194.591
30.000+ 0.213× x2 − 3.300
1.649
+0.531× x3 − 139.736
20.634,
=y = −8.655 + 0.063x1 + 0.587x2 + 0.117x3.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�¤kCþ�þjÑ���, �±��é�©êâ?1*�O:
rr.sol=lm.ridge(y∼x1+x2+x3,data=mydata,lambda=c(seq(0,0.01,by=0.001),seq(0.02,0.1,by=0.01),seq(0.2,1,by=0.1)))rr.solplot(rr.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ÀJ*ëêk = 0.4, �*£8�§:
y = −8.621 + 0.063x1 + 0.588x2 + 0.117x3.Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
2Â*�O :
PK = diag(k1, · · · , kp), ki ≥ 0, i = 1, · · · , p. ¡
β(k) = (X′X +K)−1X
′Y
�β�2Â*�O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O
̤©(Principle Component)�O´dW.F.Massyu1965cJÑ�,�«k �O, 8�´��Ñ�OÝX�¾�Ý����¦�O�½5òC�é�ù�"�. ̤©�O�Ä�g�´: Äk/Ïu��C�ò£8gCþC�éA�̤©(̤©�*ÿ�þ´pØ�'�, l �Øõ��5¯K), ,�l¤k�̤©¥À��Ü©��̤©(å�ü���^)¿±§���#�£8gCþïá#�£8�., ^���¦�{�O#�.¥�£8Xê¿��£8�§. Äu���£8�§2ò§�=���©Cþ�£8�§.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��Øþj�K�, b�gCþ�ÏCþþ®IOz.
�ģ8�.:
Y = Xβ + e, E(e) = 0, Cov(e) = σ2In, (3.9.1)
Ù¥X´n× p�OÝ. Pλ1 ≥ · · · ≥ λp > 0�X′X�A��,
φ1, · · · ,φp�éA�IO��zA��þ. K
Φ = (φ1, · · · ,φp)
�p× p��Ý�
Φ′X
′XΦ = diag(λ1, · · · , λp)
∆= Λ.
2PZ = XΦ, α = Φ′β, K�.(3.9.1)�U��
Y = Zα+ e, E(e) = 0, Cov(e) = σ2In. (3.9.2)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
3þã��5£8;K�.(3.9.2)¥, #��OÝ
Z∆= (z1, · · · , zp) = (Xφ1, · · · ,Xφp),
ePX = (x1, · · · ,xp), K#gCþ�*ÿ�þ��©gCþ�*ÿ�þ�'X�
zj = φ1jx1 + · · ·+ φpjxp, j = 1, · · · , p.
ù´é�©gCþ�*ÿ�þ����5Cþ, C��Xê�þ�A��λj¤éA�IO��zA��þ.
ÚOþ, ¡*ÿ�þzj , j = 1, · · · , péA�#gCþzj , j = 1, · · · ,p�p�̤©. z�̤©Ñ´�©gCþ��5|Ü:
zj = φ1jx1 + · · ·+ φpjxp, j = 1, · · · , p.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�5� :?¿ü��̤©�*ÿ�þÑ´pØ�'�,�1j�̤©� �²�Ú
∑ni=1(zij − zj)2 = λj .
y²: Ï�Z′Z = Φ
′X
′XΦ = Λ = diag(λ1, · · · , λp), ¤±
z′jzk = 0, ∀j 6= k
�z′jzj = λj , j = 1, · · · , p. qÏ�X´I5z�OÝ, ¤±
zj =1
n
n∑i=1
zij =1
n
n∑i=1
p∑k=1
φkjxik =1
n
p∑k=1
φkj
n∑i=1
xik = 0.
Ïdk
n∑i=1
(zij − zj)2 =
n∑i=1
z2ij = z
′jzj = λj , j = 1, · · · , p.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
λjÝþ1j�̤©zj���CÄ��. Ï�λ1 ≥ · · · ≥ λp > 0,¤±·�¡z1�1�̤©, z2�1�̤©, · · · . ùp�̤©�*ÿ�þ´pØ�'�, ¤±#�gCþz1, · · · , zp(½�OÝZ)Ø�3õ��5¯K.
d5�1��, z1 éÏCþ�)ºUå�r, z2 g�, · · · , zp�f.e�OÝX´¾�Ý, @ok�X
′X�A��é�, Ø�
b�λr+1, · · · , λp ≈ 0.
ù�, �¡�p− r�̤©���CÄÒé��þ3"NC��.¤±ùp− r�̤©éÏCþ�K�Ò�±�ÑK, �ò§�l£8�.¥GØ. ^���¦{é�e�r�̤©(=r�#�gCþ)�£8=�. ��2C£��©Cþ�£8�§.
5:̤©£8�Ì�8�: ��(�Øõ��5), ü�(~�O�þ), ,�ïá£8�§.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éΛ,α,Z,Φ�©¬:
Λ =
(Λ1 00 Λ2
), α =
(α1
α2
), Z = (Z1
...Z2), Φ = (Φ1...Φ2),
Ù¥Λ1�r × rÝ, α1�r × 1�þ, Z1�n× rÝ, Φ1�p× rÝ. GØZ2α2, �.(3.9.2)C�
Y ≈ Z1α1 + e, E(e) = 0, Cov(e) = σ2In. (3.9.3)
Z1Ø´¾�Ý, ¤±���A^���¦�O�α1�LSE
α1 = (Z1′Z1)−1Z1
′Y .
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
c¡·�l�.¥GØ�¡�p− r�̤©, ù��u·�^α2 = 0��Oα2. |^'Xβ = Φα, �β�̤©�O�
β = Φ
(α1
α2
)= (Φ1,Φ2)
(α1
0
)= Φ1Λ
−11 Z
′1Y
= Φ1Λ−11 Φ
′1X
′Y .
�A�̤©£8�§�Y = Xβ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O�L§ :
Step1:���C�Z = XΦ, ¼�#�gCþ, ¡�̤©.
Step2:�£8gCþÀJ, GØéA�A��'���@̤©.
Step3:ò�{�̤©éy����¦£8, 2�£��5�ëê�O, ��'u�©Cþ�̤©£8�§.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�O�5� :
5�1. β = Φ1Φ′1β, =̤©�O´���¦�O����5C
�.
y²: �âe�'X:
Z = (Z1...Z2) = (XΦ1
...XΦ2), Φ′1Φ1 = Ir, Φ
′1Φ2 = 0
9X
′X = ΦΛΦ
′= Φ1Λ1Φ
′1 + Φ2Λ2Φ
′2
¿5¿�X′(I −H) = 0(H�lfÝ), ��
β = Φ1Λ−11 Φ
′1X
′Y = Φ1Λ
−11 Φ
′1X
′Xβ
= Φ1Λ−11 Φ
′1Φ1Λ1Φ
′1β + Φ1Λ
−11 Φ
′1Φ2Λ2Φ
′2β
= Φ1Λ−11 Φ
′1Φ1Λ1Φ
′1β
= Φ1Φ′1β.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5�2. E(β) = Φ1Φ′1β. =��r < p, ̤©�OÒ´k �O.
y²: �I5¿�E(β) = β=�.
5�3. ‖β‖ ≤ ‖β‖, =̤©�O´Ø �O.
y²: -I = diag(Ir,0), KdΦ�½Â�
Φ1Φ′1 = ΦIΦ
′.
l k
‖β‖ = ‖ΦIΦ′β‖ = ‖IΦ
′β‖ ≤ ‖Φ′
β‖ = ‖β‖.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½n (3.9.1)
��©gCþ�3vî�õ��5�, ·�ÀJ�3�̤©�ê�¦Ì¤©�O'���¦�Ok���þ�Ø�, =
MSE(β) < MSE(β).
y²: b�X′X��p− r�A��λr+1, · · · , λpé�Cu". Ø
JwÑ
MSE(β) = MSE
(α1
0
)= tr
[Cov
(α1
0
)]+∥∥∥E
(α1
0
)−α
∥∥∥2
= σ2tr(Λ−11 ) + ‖α2‖2.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�
MSE(β) = σ2tr(Λ−1) = σ2tr(Λ−11 ) + σ2tr(Λ−1
2 ),
¤±MSE(β) = MSE(β) + (‖α2‖2 − σ2tr(Λ−1
2 )).
u´MSE(β) < MSE(β)
��=�
‖α2‖2 < σ2tr(Λ−12 ) = σ2
p∑i=r+1
1
λi. (3.9.4)
�õ��5vî��ÿ, λr+1, · · · , λp�±¿©�Cu".Ïdþªmà�±v�¦�Ø�ª(3.9.4)¤á.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
Ï�α2 = Φ′2β, (3.9.4)���(β
σ
)′
Φ2Φ2′(βσ
)< tr(Λ−1
2 ), (3.9.5)
ùÒ´`, �βÚσ÷v(3.9.5)�, ̤©�Oâ'���¦�Ok���þ�Ø�. (3.9.5)L«ëê�m¥(Àβ/σ�ëê)��¥%3�:�ý¥. u´l(3.9.5)��Xe(Ø:
(a)é�½�ëêβÚσ2, �X′X��p− r�A��'���, Ì
¤©�O'���¦�Ok���þ�Ø�.
(b)é�½�X′X, =�½�Λ2, é�é���β/σ, ̤©�O
'���¦�Ok���þ�Ø�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
̤©�êr�À� :
(1)Ñ�A���Cu"�@̤©.
(2)ÀJr¦�cr�A���Ú3p�A��oÚ¥¤Ó�'~(¡�\O�zÇ)��ýk�½��. �X, ÀJ���r¦�∑r
i=1 λi∑pi=1 λi
> 0.85.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
~3.9.1 (Y~3.8.1){I²Lêâ©Û¯K.
k?1~1����¦�O�õ��5�ä:
yx=read.table(”ex p68 data.txt”)x1=yx[, 1]x2=yx[, 2]x3=yx[, 3]y=yx[, 4]economy=data.frame(x1,x2,x3,y)economylm.sol=lm(y∼x1+x2+x3,data=economy)summary(lm.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
X=cbind(x1,x2,x3)rho=cor(X)rholibrary(DAAG)vif(lm.sol)eigen(rho)kappa(rho,exact=TRUE)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
��Øõ��5�K�, �̤©£8:
economy.pr=princomp(∼x1+x2+x3,data=economy,cor=TRUE)summary(economy.pr,loadings=TRUE)
1n�A��λ3 = 0.05187378392 = 0.00269 ≈ 0.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
éA�n�IO��zA��þ�
φ1 = (−0.706, 0,−0.707)′,
φ2 = (0,−0.999, 0)′,
φ3 = (0.707, 0,−0.707)′.
n�̤©©O�
z1 = −0.706x1 − 0.707x3,
z2 = −0.999x2,
z3 = 0.707x1 − 0.707x3.
Ï�1��A���\O�zÇ�0.666385 ≤ 0.85, cü�A���\O�zÇ0.9991030 > 0.85, ¤±·�í�1n�̤©,��3cü�̤©.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
O�̤©�©(=#Cþ�*ÿ��þ):
pre=predict(economy.pr)pre
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
?1̤©�O:
z1=pre[, 1];z2=pre[, 2]yxs=scale(yx)y=yxs[, 4]mydata=data.frame(y,z1,z2)pc.sol=lm(y∼0+z1+z2,data=mydata)summary(pc.sol)
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
�̤©£8�§:
u = −0.65787z1 − 0.1824z2
= −0.65787× (−0.706x∗1 − 0.707x∗3)− 0.1824× (−0.999x∗2)
= 0.46446x∗1 + 0.18222x∗2 + 0.46511x∗3.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
5¿±þ�IOzCþ�£8�§. =z��©Cþ�£8�§,�
y − 21.891
4.544= 0.46446× x1 − 194.591
30.000+ 0.18222× x2 − 3.300
1.649
+0.46511× x3 − 139.736
20.634,
=y = −7.768 + 0.070x1 + 0.502x2 + 0.102x3.
ùp�y, x1, x2, x3L«�©Cþ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
eL�Ñ���¦�O!*�OṲ́©�O�'�:
�{ ~ê� x1 x2 x3
���¦�O -10.128 -0.051 0.587 0.287
*�O(k = 0.04) -8.655 0.063 0.587 0.117
̤©�O(r = 2) -7.768 0.070 0.502 0.102
o�5`, *�OṲ́©�O'��C. ����¦�O�',*�OṲ́©�OÑ�ؽ�)õ��5¤�5�K�,¤±x1�£8Xê�ÎÒ�u)Cz.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SteinØ �O
*�OṲ́©�Oùü«k �O´é���¦�Oβ��:�·��Ø . �¦�éëê�Oβ��©þ��´�þ!Ø .�!òÄuStein�Ø g�?Ø���¦�Oβ�þ!Ø �O, T�O´Stein31955cJÑ5�.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
½Â (Stein�O)
3�5£8�.(3.1.5)¥, Ù£8Xêβ����¦�O�β =(X
′X)−1X
′Y . ·�¡
βs(c) = cβ
�Stein�O, Ù¥0 ≤ c ≤ 1�¡�Ø Xê.
w,, βs(c)´éβ�z�©þ����Ø , ¤±Stein�O´�«þ!Ø �O.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
SteinØ �O�5� :
5�1.�c 6= 1�, βs(c)´β�k !Ø �O.
5�2.�30 < c < 1, ¦�MSE(βs(c)) < MSE(β).
y²: βs(c)�þ�Ø��
MSE(βs(c)) = tr[Cov(βs(c))] + ‖E(βs(c))− β‖2
= c2σ2tr[(X′X)−1] + (c− 1)2‖β‖2
= c2σ2p∑i=1
λ−1i + (c− 1)2‖β‖2
∆= g(c).
ùp, ��Øþj�K�, ·�b�gCþÚÏCþþ®IOz, Ïd�OÝX�n× pÝ.
Tianxiao Pang 1nÙ £8ëê��O
LSE LSE�5� �åLSE �ä BCC� GLSE õ��5 *�O PC�O Stein
ég(c)'uc¦�¿-Ù�u"�)�c��`��
c∗ =‖β‖2
σ2∑p
i=1 λ−1i + ‖β‖2
< 1. (3.10.1)
dug(c)'uc����ê�u
2σ2p∑i=1
λ−1i + 2‖β‖2 > 0,
Ïdg(c) = MSE(βs(c))3c∗?����, ¿��c∗ ≤ c < 1�,
kMSE(βs(c)) < MSE(β).
Tianxiao Pang 1nÙ £8ëê��O