1
Hypothesis Testing UnderGeneral Linear Model
Previously we derived the sampling property results assuming normality:Y = X + e where et~N(0,2)→ Y~N(X,2IT)s=(X'X)-1X'Y, E(s)=Cov(s)= β =2(X'X)-1
l~N(, 2(X'X)-1)σU
2 unbiased estimate of σ2
An estimate of Cov(βs) = βs=σU
2(X'X)-1
2 l lU
e eσ =
(T-K)
el = y - Xβl
2
Hypothesis Testing UnderGeneral Linear Model
Single Parameter (βk,L) Hypothesis Test βk,l~N(βk,Var(βk))
kth diagonal element of βs
When σ2 is known:
( )
, ~ (0,1)var
k l k
k
z Nb b
b
-=
unknown true coeff.
When σ2 not known:
( )
, ~ˆvar
k l kT K
k
t tb b
b-
-=
Σβs=σu2(X'X)-1
3
Hypothesis Testing UnderGeneral Linear Model
Can obtain (1-) CI for βk:
There is a (1-α) probability that the true unknown value of β is within this range
Does this interval contain our hypothesized value? If it does, than we can not reject H0
( )
( )
k,l k α 2,T-K k
k,l k α 2,T-K
ˆβ - var β t β
ˆβ + var β t
£
£
4
Hypothesis Testing UnderGeneral Linear Model
Testing More Than One Linear Combination of Estimated Coefficients Assume we have a-priori
information about the value of β
We can represent this information via a set of J-Linear hypotheses (or restrictions):
In matrix notation
K
jk k jk=1
R β = r j=1,2,…,J (J K)
(JxK) (Jx1)(Kx1)R β = r
5
Hypothesis Testing UnderGeneral Linear Model
(JxK) (Jx1)(Kx1)R β = r
11 12 1
21 22 2
1 2
K
k
J J JK
R R R
R R RR
R R R
1
2
J
r
rr
r
knowncoefficients
6
Hypothesis Testing UnderGeneral Linear Model
Assume we have a model with 5 parameters to be estimatedJoint hypotheses: β1=8 and β2=β3
J=2, K=5
0
1
2
3
4
0 1 0 0 0 8
0 0 1 1 0 0
0 1 0 0 0 8
0 0 1 1 0 0
R r
R r
β2-β3=0
7
Hypothesis Testing UnderGeneral Linear Model
How do we obtain parameter estimates if J hypotheses are true? Constrained (Restricted) Least
Squares R is β that minimizes: S=(Y-Xβ)'(Y-Xβ) s.t. Rβ=r
= e'e s.t. Rβ=re.g. we act as if H0 are true
S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ) λ is (J x1) Lagrangian multipliers
associated with J-joint hypotheses We want to choose β such that we
minimize SSE but also satisfy the J constraints (hypotheses), βR
8
Hypothesis Testing UnderGeneral Linear Model
Min. S*=(Y-Xβ)'(Y-Xβ) + λ'(r-Rβ)
What and how many FOC’s? K+J FOC’s
1
* * *0
K
S S S
K-FOC’s
*0
Sr R
J-FOC’s
9
Hypothesis Testing UnderGeneral Linear Model
What are the FOC’s?
( 1)
-1 -1R
*-2X Y + 2X Xβ - R λ 0
move -2X Y and -R λ to RHS
and divide by 2
R λX Xβ = X Y +
2R λ
β = X X X Y + X X2
Kx
S
Substitute these FOC into 2nd set∂S*/∂λ = (r-RβR) = 0J →
-1sR λ
r = Rβ + R X X2
S*=(Y-Xβ)'(Y-Xβ)+λ'(r-Rβ)
CRM
βS
10
Hypothesis Testing UnderGeneral Linear Model
-1s
-1-1s
λr-Rβ = R X X R 2
λ = R X X R r-Rβ2
The 1st FOC
Substitute the expression for λ/2 into the 1st FOC:
R s
1-1 -1sX X R R X X R r-Rβ
1 1 RX X X Y X X
2R
11
Hypothesis Testing UnderGeneral Linear Model
βR is the restricted LS estimator of β as well as the restricted ML estimator
Properties of Restricted Least Squares Estimator
→E(R) if R rV(R) ≤ V(S)
→[V(S) - V(R)] is positive semi-definite
diag(V(R)) ≤ diag(V(S))
R
1-1 -1
E
X X R R X X R r-Rβ
True butUnknown Value
12
Hypothesis Testing UnderGeneral Linear Model
From above, if Y is multivariate normal and H0 is true βl,R~N(β,σ2M*(X'X)-1M*')
~N(β,σ2M*(X'X)-1)
From previous results, if r-Rβ≠0 (e.g., not all H0 true), estimate of β is biased if we continue to assume r-Rβ=0
,
11 1
Bias l RE
X X R R X X R r R
≠0
11 1*
KM I X X R R X X RR
13
Hypothesis Testing UnderGeneral Linear Model
The variance is the same regardless of he correctness of the restrictions and the biasedness of βR → βR has a variance that is smaller
when compared to βs which only uses the sample information.
14
Hypothesis Testing UnderGeneral Linear Model
Beer Consumption Example : qB ≡ quantity of beer purchased
PB ≡ price of beerPL ≡ price of other alcoholic bev.PO≡ price of other goodsINC ≡ household income
Real Prices Matter? All prices and INC by 10% β1 + β2 + β3 + β4=0
Equal Price Impacts? Liquor and Other Goods β2=β3
Unitary Income Elasticity? β4=1
Data used in the analysis
ββ β β31 2 4B B L Oq =αP P P INC exp( )e
15
Given the above, what does the R-matrix and r vector look like for these joint tests?
Lets develop a test statistic to test these joint hypotheses
We are going to use the Likelihood Ratio (LR) to test the joint hypotheses
Hypothesis Testing UnderGeneral Linear Model
0 1 1 1 1 0
0 0 1 1 0 0
0 0 0 0 1 1
R r
B,t 0 1 B,t 2 L,t
3 O,t 4 t t
lnq =β +β ln(P )+β ln(P )
+β ln(P )+β ln(Inc )+e
16
Hypothesis Testing UnderGeneral Linear Model
LR=lU*/lR
*
lU*=Max [l(|y1,…,yT);
=(β, σ) ]= “unrestricted” maximum likelihood function
lR*=Max [l(|y1,…,yT);
=(β, σ); Rβ=r]= “restricted” maximum likelihood function
Again, because we are possibly restricting the parameter space via our null hypotheses, LR≥1
17
Hypothesis Testing UnderGeneral Linear Model
If lU* is large relative to lR
*→data shows evidence that the restrictions (hypotheses) are not true (e.g., reject null hypothesis) How much should LR exceed 1
before we reject H0? We reject H0 when LR ≥ LRC where
LRC is a constant chosen on the basis of the relative cost of the Type I vs. Type II errors
When implementing the LR Test you need to know the PDF of the dependent variable which determines the density of the test statistic
18
Hypothesis Testing UnderGeneral Linear Model
For LR test, assume Y has a normal distribution →e~N(0,σIT) This implies the following LR
test statistic (LR*) What are the distributional
characteristics of LR*? Will address this in a bit
19
Hypothesis Testing UnderGeneral Linear Model
We can derive alternative specifications of LR test statistic LR*=(SSER-SSEU)/(J2U)
(ver. 1) LR*=[(Re-r)′[R(X′X)-1R′]-1(Re-r)]/(J2U)
(ver. 2) LR*=[(R-e)′(X′X)(R-e)]/(J2U)
(ver. 3)βe =βS=βl
What are the Distributional Characteristics of LR* (JHGLL p. 255) LR* ~ FJ,T-K
J = # of Hypotheses K= # of Parameters (including intercept)
20
Hypothesis Testing UnderGeneral Linear Model
Proposed Test Procedure Choose = P(reject H0| H0 true) =
P(Type-I error) Calculate the test statistic LR*
based on sample information Find the critical value LRcrit in an F-
table such that: = P(F(J, T – K) LRcrit), where α =
P(reject H0| H0 true)f(LR*)
αLRcrit
α = P(FJ,T-K ≥ LRcrit)
21
Hypothesis Testing UnderGeneral Linear Model
Proposed Test Procedure Choose = P(reject H0| H0 true) =
P(Type-I error) Calculate the test statistic LR*
based on sample information Find the critical value LRcrit in an F-
table such that: = P(F(J, T – K) LRcrit), where α =
P(reject H0| H0 true) Reject H0 if LR* LRcrit
Don’t reject H0 if LR* < LRcrit
22
Hypothesis Testing UnderGeneral Linear Model
Beer Consumption Example
Does the regression do a better job in explaining variation in beer consumption than if assumed the mean response across all obs.? Remember SSE=(T-K)σ2
U
Under H0: All slope coefficients=0
Under H0, TSS=SSE given that that there is no RSS and TSS=RSS+SSE
B,t 0 1 B,t 2 L,t
3 O,t 4 t t
lnq =β +β ln(P )+β ln(P )
+β ln(P )+β ln(Inc )+e
23
Hypothesis Testing UnderGeneral Linear Model
Log-Log Beer Consumption Model
Unconstrained Model
R2 0.8254
Adj. R2 0.7975
σU 0.05997
Obs 30
Variable Coeff Std Error T-Stat
Intercept -3.243 3.743 -0.87
lnPB -1.020 0.239 -4.27
lnPL -0.583 0.560 -1.04
lnPO 0.210 0.080 2.63
ln(INC) 0.923 0.416 2.22
Constrained Model
σU 0.13326 SSER=0.133262*29=0.51497
Coeff Std Error T-Stat
Intercept 4.019 0.0243 165.17
SSE = 0.059972 *25 = 0.08992
R2=1- 0.08992/0.51497
TSS=SSERMean of LN(Beer)
24
Hypothesis Testing UnderGeneral Linear Model
Results of our test of overall significance of regression model
Lets look at the following GAUSS Code
GAUSS command:CDFFC(29.544,4,25)=3.799e-009CDFFC Computes the complement
of the cdf of the F distribution (1-Fdf1,df2)
Unlikely value of F if hypothesis is true, that is no impact of exogenous variables on beer consumption
Reject the null hypothesisAn alternative look
25
Hypothesis Testing UnderGeneral Linear Model
Beer Consumption Example
Three joint hypotheses exampleSum of Price and Income
Elasticities Sum to 0 (e.g., β1 + β2 + β3 + β4=0)Other Liquor and Other Goods
Price Elasticities are Equal (e.g., β2=β3)
Income Elasticity = 1 (e.g., β4=1) cdffc(0.84,3,25)=0.4848
B,t 0 1 B,t 2 L,t
3 O,t 4 t t
lnq =β +β ln(P )+β ln(P )
+β ln(P )+β ln(Inc )+e
26
Hypothesis Testing UnderGeneral Linear Model
F3,25
0.84
area = 0.4848
Location of our calculated test statistic
F
27
Hypothesis Testing UnderGeneral Linear Model
A side note: How do you estimate the variance of an elasticity and therefore test H0 about this elasticity?
Suppose you have the following model:FDXt = β0 + β1Inct + β2 Inc2
t + et
FDX= food expenditure Inc=household income
Want to estimate the impacts of a change in income on expenditures. Use an elasticity measure evaluated at mean of the data. That is:
28
Hypothesis Testing UnderGeneral Linear Model
Income Elasticity (Γ) is:
How do you calculate the variance of Γ?
We know that: Var(α′Z)= α′Var(Z)α Z is a column vector of RV’s α a column vector of constants
Treat β0, β1 and β2 are RV’s. The α vector is:
1 2FDX Inc Inc
2 IncInc FDX FDX
2Inc 2Inc
α 0FDX FDX
FDXt = β0 + β1Inct + β2 Inc2t + et
Linear combination of Z
29
Hypothesis Testing UnderGeneral Linear Model
This implies var(Γ) is:
2 0 0 1 0 2
0 1 1 1 2
0 2 1 2 2 2
0Var β Cov β ,β Cov β ,β
Inc 2Inc Inc0 Cov β ,β Var β Cov β ,β
FDX FDX FDXCov β ,β Cov β ,β Var β
2Inc
FDX
21 1 2
21 2 2
(2 2)(1 2)
(2 1)
Inc
Var β Cov β ,β FDXInc 2Inc
Cov β ,β Var βFDX FDX 2Inc
FDXxx
x
(1 x 1)
σ2(X'X)-1
(3 x 3)
(1 x 3)
(3 x 1)Due to 0 α value
30
Hypothesis Testing UnderGeneral Linear Model
This implies: var(Γ) is:
22 2
1 2
2
1 2
Inc 2IncVar Var β Var β
FDX FDX
Inc 2Inc2 Cov β ,β
FDX FDX
C12
C22
22 2
1 2
3
1 22
Inc 2IncVar β Var β
FDX FDX
Inc4 Cov β ,β
FDX
2C1C2