1、第1页，共69页。yt =household weekly food expendituresSimple Linear Regression Modelyt =b1+b2 x t +e tx t =household weekly incomeFor a given level of x t,the expectedlevel of food expenditures will be:E(yt|x t)=b1+b2 x t4.2第2页，共69页。1.yt =b1+b2x t +e t2.E(e t)=0 E(yt)=b1+b2x t 3.var(e t)=s 2 =var(yt)4.cov(
2、e i,e j)=cov(yi,yj)=05.x t c for every observation6.e tN(0,s 2)ytN(b1+b2x t,s 2)Assumptions of the SimpleLinear Regression Model4.3第3页，共69页。The population parameters b1 and b2are unknown population constants.The formulas that produce thesample estimates b1 and b2 arecalled the estimators of b1 and b
3、2.When b0 and b1 are used to representthe formulas rather than specific values,they are called estimators of b1 and b2which are random variables becausethey are different from sample to sample.4.4第4页，共69页。If the least squares estimators b0 and b1are random variables,then what are theirmeans,variance
4、s,covariances andprobability distributions?Compare the properties of alternative estimators to the properties of the least squares estimators.Estimators are Random Variables(estimates are not)4.5第5页，共69页。The Expected Values of b1 and b2 The least squares formulas(estimators)in the simple regression
5、case:b2=nSxiyi-Sxi SyinSxi2-(Sxi)2 2b1=y -b2xwhere y=Syi/n and x=Sx i/n (4.1a)(4.1b)4.6第6页，共69页。Substitute in yi =b1+b2xi +e ito get:b2=b2 +nSxiei-Sxi SeinSxi-(Sxi)22The mean of b2 is:Eb2=b2 +nSxiEei-Sxi SEeinSxi-(Sxi)22Since Eei=0,then Eb2=b2.4.7第7页，共69页。The result Eb2=b2 means thatthe distribution
6、 of b2 is centered at b2.Since the distribution of b2 is centered at b2,we say thatb2 is an unbiased estimator of b2.An Unbiased Estimator 4.8第8页，共69页。The unbiasedness result on the previous slide assumes that weare using the correct model.If the model is of the wrong formor is missing important var
7、iables,then Eei=0,then Eb2=b2.Wrong Model Specification 4.9第9页，共69页。Unbiased Estimator of the Intercept In a similar manner,the estimator b1of the intercept or constant term can beshown to be an unbiased estimator of b1 when the model is correctly specified.Eb1=b14.10第10页，共69页。b2=nSxiyi-Sxi SyinSxi-
8、(Sxi)22(4.3b)(4.3a)Equivalent expressions for b2:Expand and multiply top and bottom by n:b2=S(xi-x)(yi-y)S(xi-x)24.11第11页，共69页。Variance of b2 Given that both yi and ei have variance s s 2,the variance of the estimator b2 is:b2 is a function of the yi values butvar(b2)does not involve yi directly.S(x
9、 i-x)s s 22var(b2)=4.12第12页，共69页。Variance of b1 nS(x i-x)2var(b1)=s s 2Sx i2the variance of the estimator b1 is:b1=y -b2xGiven4.13第13页，共69页。Covariance of b1 and b2 S(x i-x)2cov(b1,b2)=s s2-x If x=0,slope can change without affectingthe variance.4.14第14页，共69页。What factors determine variance and covar
10、iance?1.s s 2:uncertainty about yi values uncertainty about b1,b2 and their relationship.2.The more spread out the xi values are then the more confidence we have in b1,b2,etc.3.The larger the sample size,n,the smaller the variances and covariances.4.The variance b1 is large when the(squared)xi value
11、s are far from zero(in either direction).5.Changing the slope,b2,has no effect on the intercept,b1,when the sample mean is zero.But if sample mean is positive,the covariance between b1 and b2 will be negative,and vice versa.4.15第15页，共69页。Gauss-Markov Theorem Under the first five assumptions of the s
12、imple,linear regression model,the ordinary least squares estimators b1 and b2 have the smallest variance of all linear and unbiased estimators of b1 and b2.This means that b1and b2 are the Best Linear Unbiased Estimators (BLUE)of b1 and b2.4.16第16页，共69页。implications of Gauss-Markov1.b1 and b2 are“be
13、st”within the class of linear and unbiased estimators.2.“Best”means smallest variance within the class of linear/unbiased.3.All of the first five assumptions must hold to satisfy Gauss-Markov.4.Gauss-Markov does not require assumption six:normality.5.G-Markov is not based on the least squares princi
14、ple but on b1 and b2.4.17第17页，共69页。G-Markov implications(continued)6.If we are not satisfied with restricting our estimation to the class of linear and unbiased estimators,we should ignore the Gauss-Markov Theorem and use some nonlinear and/or biased estimator instead.(Note:a biased or nonlinear est
15、imator could have smaller variance than those satisfying Gauss-Markov.)7.Gauss-Markov applies to the b1 and b2 estimators and not to particular sample values(estimates)of b1 and b2.4.18第18页，共69页。Probability Distribution of Least Squares Estimators b2 N b2,S(xi-x)s s 22b1 N b1,nS(x i-x)2s s 2 Sxi24.1
16、9第19页，共69页。yi and e i normally distributed The least squares estimator of b2 can beexpressed as a linear combination of yis:b2=S wi yi b1=y -b2x S(x i-x)2where wi=(x i-x)This means that b1and b2 are normal sincelinear combinations of normals are normal.4.20第20页，共69页。normally distributed under The Ce
17、ntral Limit TheoremIf the first five Gauss-Markov assumptionshold,and sample size,n,is sufficiently large,then the least squares estimators,b1 and b2,have a distribution that approximates thenormal distribution with greater accuracythe larger the value of sample size,n.4.21第21页，共69页。Consistency We w
18、ould like our estimators,b1 and b2,to collapse onto the true population values,b1 and b2,as sample size,n,goes to infinity.One way to achieve this consistency property is for the variances of b1 and b2 to go to zero as n goes to infinity.Since the formulas for the variances of the least squares esti
19、mators b1 and b2 show that their variances do,in fact,go to zero,then b1 and b2,are consistent estimators of b1 and b2.4.22第22页，共69页。Estimating the variance of the error term,s s 2et =yt-b1-b2 x tSett=1T2n-2s 2 2 =s 2 2 is an unbiased estimator of s 2 4.23第23页，共69页。The Least Squares Predictor,yo Giv
20、en a value of the explanatory variable,Xo,we would like to predicta value of the dependent variable,yo.The least squares predictor is:yo =b1+b2 x o (4.4)4.24第24页，共69页。Inference in the Simple Regression ModelChapter 55.1第25页，共69页。1.yt =b1+b2x t +e t2.E(e t)=0 E(yt)=b1+b2x t 3.var(e t)=s 2 =var(yt)4.c
21、ov(e i,e j)=cov(yi,yj)=05.x t c for every observation6.e tN(0,s 2)ytN(b1+b2x t,s 2)Assumptions of the Simple Linear Regression Model5.2第26页，共69页。Probability Distribution of Least Squares Estimators b1 N b1,n S(x t-x)2s s2 Sx t2b2 N b2,S(x t-x)s s225.3第27页，共69页。s 2=n-2et2 2SUnbiased estimator of the
22、error variance:s 2s 2(n-2)n-2cTransform to a chi-square distribution:Error Variance Estimation 5.4第28页，共69页。We make a correct decision if:The null hypothesis is false and we decide to reject it.The null hypothesis is true and we decide not to reject it.Our decision is incorrect if:The null hypothesi
23、s is true and we decide to reject it.This is a type I error.The null hypothesis is false and we decide not to reject it.This is a type II error.5.5第29页，共69页。b2 N b2,S(x t-x)s s22Create a standardized normal random variable,Z,by subtracting the mean of b2 and dividing by its standard deviation:b2-b2
24、var(b2)Z =N(0,1)5.6第30页，共69页。Simple Linear Regressionyt =b1+b2x t +e t where E e t=0yt N(b1+b2x t,s 2)since Eyt=b1+b2x t e t =yt -b1-b2x t Therefore,e t N(0,s 2).5.7第31页，共69页。Create a Chi-Squaree t N(0,s 2)but want N(0,1).(e t/s)N(0,1)Standard Normal.(e t/s)2 c2(1)Chi-Square.5.8第32页，共69页。Sum of Chi-
25、SquaresSt=1(e t/s)2=(e1/s)2+(e 2/s)2+.+(e n/s)2 c2(1)+c2(1)+.+c2(1)=c2(N)Therefore,St=1(e t/s)2 c2(N)5.9第33页，共69页。Since the errors e t =yt -b1-b2x t are not observable,we estimate them with the sample residuals e t =yt -b1-b2x t.Unlike the errors,the sample residuals arenot independent since they us
26、e up two degrees of freedom by using b1 and b2 to estimate b1 and b2.We get only n-2 degrees of freedom instead of n.Chi-Square degrees of freedom5.10第34页，共69页。Student-t Distributiont=t(m)ZV/mwhere Z N(0,1)and V c(m)25.11第35页，共69页。t =t(n-2)ZV/(n-2)where Z=(b2-b2)var(b2)and var(b2)=s 2S(xi-x)25.12第36
27、页，共69页。t =ZV/(n-2)(b2-b2)var(b2)t =(n-2)s 2s 2(n-2)V =(n-2)s 2s 25.13第37页，共69页。var(b2)=s 2S(xi-x)2(b2-b2)s 2S(xi-x)2t =(n-2)s 2s 2(n-2)(b2-b2)s 2S(xi-x)2 notice thecancellations5.14第38页，共69页。(b2-b2)s 2S(xi-x)2t =(b2-b2)var(b2)t =(b2-b2)se(b2)5.15第39页，共69页。Students t-statistic t =t(n-2)(b2-b2)se(b2)t
28、 has a Student-t Distribution with n-2 degrees of freedom.5.16第40页，共69页。Figure 5.1 Student-t Distribution(1-a)t0f(t)-tctca/2a/2red area=rejection region for 2-sided test5.17第41页，共69页。probability statementsP(-tc t tc)=1-aP(t tc)=a/2 2P(-tc tc)=1-a(b2-b2)se(b2)5.18第42页，共69页。Confidence IntervalsTwo-sid
29、ed(1-a)x100%C.I.for b1:b1-ta/2se(b1),b1+ta/2se(b1)b2-ta/2se(b2),b2+ta/2se(b2)Two-sided(1-a)x100%C.I.for b2:5.19第43页，共69页。Student-t vs.Normal Distribution1.Both are symmetric bell-shaped distributions.2.Student-t distribution has fatter tails than the normal.3.Student-t converges to the normal for in
30、finite sample.4.Student-t conditional on degrees of freedom(df).5.Normal is a good approximation of Student-t for the first few decimal places when df 30 or so.5.20第44页，共69页。Hypothesis Tests1.A null hypothesis,H0.2.An alternative hypothesis,H1.3.A test statistic.4.A rejection region.5.21第45页，共69页。Re
31、jection Rules1.Two-Sided Test:If the value of the test statistic falls in the critical region in either tail of the t-distribution,then we reject the null hypothesis in favor of the alternative.2.Left-Tail Test:If the value of the test statistic falls in the critical region which lies in the left ta
32、il of the t-distribution,then we reject the null hypothesis in favor of the alternative.2.Right-Tail Test:If the value of the test statistic falls in the critical region which lies in the right tail of the t-distribution,then we reject the null hypothesis in favor of the alternative.5.22第46页，共69页。Fo
33、rmat for Hypothesis Testing1.Determine null and alternative hypotheses.2.Specify the test statistic and its distribution as if the null hypothesis were true.3.Select a and determine the rejection region.4.Calculate the sample value of test statistic.5.State your conclusion.5.23第47页，共69页。practical vs
34、.statistical significance in economicsPractically but not statistically significant:When sample size is very small,a large average gap between the salaries of men and women might not be statistically significant.Statistically but not practically significant:When sample size is very large,a small cor
35、relation(say,r=0.00000001)between the winning numbers in the PowerBall Lottery and the Dow-Jones Stock Market Index might be statistically significant.5.24第48页，共69页。Type I and Type II errorsType I error:We make the mistake of rejecting the null hypothesis when it is true.a=P(rejecting H0 when it is
36、true).Type II error:We make the mistake of failing to reject the null hypothesis when it is false.b=P(failing to reject H0 when it is false).5.25第49页，共69页。Prediction IntervalsA(1-a)x100%prediction interval for yo is:yo tc se(f)se(f)=var(f)f =yo-yoS(x t-x)2var(f)=s s 2 1+1n(x o-x)25.26第50页，共69页。Two-s
37、ided Hypothesis5.275.6 HYPOTHESIS TESTING:1 THE CONFIDENCE-INTERVAL APPROACH第51页，共69页。5.28One-Sided or One-Tail TestH0:2 0.3 and H1:2 0.3Sometimes we have a strong a priori or theoretical expectation(or expectations based on some previous empirical work)that the alternative hypothesis is one-sided o
38、r unidirectional rather than two-sided,as just discussed.Thus,for our consumptionincome example,one could postulate that H0:2 0.3 and H1:2 0.3Perhaps economic theory or prior empirical work suggests that the marginal propensity to consume is greater than 0.3.Although the procedure to test this hypot
39、hesis can be easily derived from earlier works,the actual mechanics are better explained in terms of the test-of-significance approach discussed next.第52页，共69页。5.29 Before concluding our discussion of hypothesis testing,note that the testing procedure just outlined is known as a two-sided,or two-tai
40、l,test-of-significance procedure in that we consider the two extreme tails of the relevant probability distribution,the rejection regions,and reject the nullhypothesis if it lies in either tail.But this happens because our H1 was a two-sided composite hypothesis;2=0.3 means 2 is either greater than
41、or less than 0.3.But suppose prior experience suggests to us that the MPC is expected to be greater than 0.3.In this case we have:H0:2 0.3 and H1:2 0.3.Although H1 is still a composite hypothesis,It is now one-sided.第53页，共69页。To test this hypothesis,we use the one-tail test(the right tail),as shown
42、in Figure 5.5.The test procedure is the same as before except that the upper confidence limit or critical value now corresponds to t=t.05,that is,the 5 percent level.As Figure 5.5 shows,we need not consider the lower tail of the t distribution in this case.Whether one uses a two-or one-tail test of
43、significance will depend upon how the alternative hypothesis is formulated,which,in turn,may depend upon some a priori considerations or prior empirical experience.5.30第54页，共69页。5.31第55页，共69页。HYPOTHESIS TESTING:2 THE TEST-OF-SIGNIFICANCE APPROACHTesting the Significance of Regression Coefficients:Th
44、e t Test An alternative but complementary approach to the confidence-intervalmethod of testing statistical hypotheses is the test-of-significance approach developed along independent lines by R.A.Fisher and jointly by Neyman and Pearson.Broadly speaking,a test of significance is a procedure by which
45、 sample results are used to verify the truth or falsity of a null hypothesis.The key idea behind tests of significance is that of a test statistic(estimator)and the sampling distribution of such a statistic under the null hypothesis.The decision to accept or reject H0 is made on the basis of the val
46、ue of the test statistic obtained from the data at hand.As an illustration,recall that under the normality assumption the variable follows the t distribution with n-2 df.If the value of true 2 is specifiedunder the null hypothesis,the t value can readily be computedfrom the available sample,and ther
47、efore it can serve as a test statistic.5.32第56页，共69页。5.33第57页，共69页。5.34第58页，共69页。5.35第59页，共69页。5.36第60页，共69页。APPLICATION OF REGRESSION ANALYSIS:THE PROBLEM OF PREDICTIONOn the basis of the sample data of Table 3.2 we obtained the following sample regression:Y=24.4545+0.5091Xi 5.37第61页，共69页。5.38第62页，
48、共69页。第63页，共69页。5.39第64页，共69页。5.40第65页，共69页。SUMMARYAND CONCLUSIONS 1.The basic framework of regression analysis is the CLRM(Classic Linear Regression Model).2.The CLRM is based on a set of assumptions.3.Based on these assumptions,the least-squares estimators take on certain properties summarized in t
49、he GaussMarkov theorem,which states that in the class of linear unbiased estimators,the least-squares estimators have minimum variance.In short,they are BLUE第66页，共69页。SUMMARYAND CONCLUSIONS4.The precision of OLS estimators is measured by their standard errors.to draw inferences on the population par
50、ameters,the coefficients.5.The overall goodness of fit of the regression model is measured by the coefficient of determination,r2.It tells what proportion of the variation in the dependent variable,or regressand,is explained by the explanatory variable,or regressor.This lies between 0 and 1;the clos
