1、Qualitative and Limited Dependent Variable ModelsECON 6002Econometrics Memorial University of NewfoundlandAdapted from Vera Tabakovas notes 16.1 Models with Binary Dependent Variables16.2 The Logit Model for Binary Choice16.3 Multinomial Logit16.4 Conditional Logit16.5 Ordered Choice Models16.6 Mode
2、ls for Count Data16.7 Limited Dependent VariablesSlide 16-2Principles of Econometrics,3rd EditionExamples of multinomial choice(polytomous)situations:1.Choice of a laundry detergent:Tide,Cheer,Arm&Hammer,Wisk,etc.2.Choice of a major:economics,marketing,management,finance or accounting.3.Choices afte
3、r graduating from high school:not going to college,going to a private 4-year college,a public 4 year-college,or a 2-year college.Slide16-3Principles of Econometrics,3rd EditionThe explanatory variable xi is individual specific,but does not change across alternatives.Example age of the individual.The
4、 dependent variable is nominal Slide16-4Principles of Econometrics,3rd EditionExamples of multinomial choice situations:1.It is key that there are more than 2 choices2.It is key that there is no meaningful ordering to them.Otherwise we would want to use that information(with an ordered probit or ord
5、ered logit)Slide16-5Principles of Econometrics,3rd EditionIn essence this model is like a set of simultaneous individual binomial logistic regressionsWith appropriate weighting,since the different comparisons between different pairs of categories would generally involve different numbers of observat
6、ionsSlide16-6Principles of Econometrics,3rd EditionSlide16-7Principles of Econometrics,3rd Edition(16.19a)(16.19c)(16.19b)1122213231,11expexpiiipjxx 1222212221323exp,21expexpiiiixpjxx 1323312221323exp,31expexpiiiixpjxx individual chooses alternative ijpPijSlide16-8Principles of Econometrics,3rd Edit
7、ion1122331122331222 11323 1122221222213232132331222313233122213231,1,111expexpexp1expexpexp1expexp,P yyypppxxxxxxxxL Slide16-9Principles of Econometrics,3rd Edition01122201323011expexppxx(16.20)3221all else constantimimimmjijjiippppxx1111222132312221323111expexp1expexpbabbaapppxxxx An interesting fe
8、ature of the odds ratio(16.21)is that the odds of choosing alternative j rather than alternative 1 does not depend on how many alternatives there are in total.There is the implicit assumption in logit models that the odds between any pair of alternatives is independent of irrelevant alternatives(IIA
9、).Slide16-10Principles of Econometrics,3rd Edition(16.21)(16.22)121exp2,31ijijjiiipP yjxjP yp1212exp2,3ijijjjiippxjxThere is the implicit assumption in logit models that the odds between any pair of alternatives is independent of irrelevant alternatives(IIA)One way to state the assumptionIf choice A
10、 is preferred to choice B out of the choice set A,B,then introducing a third alternative X,thus expanding that choice set to A,B,X,must not make B preferable to A.which kind of makes sense Slide16-11Principles of Econometrics,3rd EditionIIA assumptionThere is the implicit assumption in logit models
11、that the odds between any pair of alternatives is independent of irrelevant alternatives(IIA)In the case of the multinomial logit model,the IIA implies that adding another alternative or changing the characteristics of a third alternative must not affect the relative odds between the two alternative
12、s considered.This is not realistic for many real life applications involving similar(substitute)alternatives.Slide16-12Principles of Econometrics,3rd EditionIIA assumptionThis is not realistic for many real life applications with similar(substitute)alternativesExamples:Beethoven/Debussy versus anoth
13、er of Beethovens Symphonies(Debreu 1960;Tversky 1972)Bicycle/Pony(Luce and Suppes 1965)Red Bus/Blue Bus(McFadden 1974).Black slacks,jeans,shorts versus blue slacks(Hoffman,2004)Etc.Slide16-13Principles of Econometrics,3rd EditionIIA assumptionThis is not realistic for many real life applications wit
14、h similar(substitute)alternativesExamples:Beethoven/Debussy(Debreu 1960;Tversky 1972)Bicycle/Pony(Luce and Suppes 1965)Red Bus/Blue Bus(McFadden 1974).Black slacks,blue slacks,jeans,shorts(Hoffman,2004)Etc.Slide16-14Principles of Econometrics,3rd EditionIIA assumptionRed Bus/Blue Bus(McFadden 1974).
15、Imagine commuters first face a decision between two modes of transportation:car and red bus Suppose that a consumer chooses between these two options with equal probability,0.5,so that the odds ratio equals 1.Now add a third mode,blue bus.Assuming bus commuters do not care about the color of the bus
16、(they are perfect substitutes),consumers are expected to choose between bus and car still with equal probability,so the probability of car is still 0.5,while the probabilities of each of the two bus types should go down to 0.25However,this violates IIA:for the odds ratio between car and red bus to b
17、e preserved,the new probabilities must be:car 0.33;red bus 0.33;blue bus 0.33Te IIA axiom does not mix well with perfect substitutes IIA assumptionWe can test this assumption with a Hausman-McFadden test which compares a logistic model with all the choices with one with restricted choices(mlogtest,h
18、ausman base in STATA,but check option detail too:mlogtest,hausman detail)However,see Cheng and Long(2007)Another test is Small and Hsiaos(1985)STATAs command is mlogtest,smhsiao(careful:the sample is randomly split every time,so you must set the seed if you want to replicate your results)See Long an
19、d Freeses book for details and worked examples IIA assumptionIIA assumption 3 0.021 4 1.000 for Ho 2 0.206 4 0.995 for Ho Omitted chi2 df Pchi2 evidence Ho:Odds(Outcome-J vs Outcome-K)are independent of other alternatives.*Hausman tests of IIA assumption(N=1000).mlogtest,hausman _cons 4.57382 .43923
20、76 10.41 0.000 3.71293 5.43471 parcoll 1.067561 .274181 3.89 0.000 .5301758 1.604946 faminc .0132383 .0038992 3.40 0.001 .005596 .0208807 grades -.6558358 .0540845 -12.13 0.000 -.7618394 -.54983213 _cons 1.942856 .4561356 4.26 0.000 1.048847 2.836866 parcoll .5370023 .2892469 1.86 0.063 -.0299112 1.
21、103916 faminc .0080757 .004009 2.01 0.044 .0002182 .0159332 grades -.2891448 .0530752 -5.45 0.000 -.3931703 -.18511922 1 (base outcome)psechoice Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-847.54576 Pseudo R2 =0.1680 Prob chi2 =0.0000 LR chi2(6)=342.22Multinomial logistic regression Number o
22、f obs =1000.mlogit psechoice grades faminc parcoll,baseoutcome(1)nologuse nels_small,clearaverage grade on 13 point scale with 1=highestIIA assumption.3 -156.227 -153.342 5.770 3 0.123 for Ho 2 -171.559 -170.581 1.955 3 0.582 for Ho Omitted lnL(full)lnL(omit)chi2 df Pchi2 evidence Ho:Odds(Outcome-J
23、vs Outcome-K)are independent of other alternatives.*Small-Hsiao tests of IIA assumption(N=1000).mlogtest,smhsiao _cons 4.724423 .4362826 10.83 0.000 3.869325 5.579521 faminc .0188675 .0037282 5.06 0.000 .0115603 .0261747 grades -.6794793 .0535091 -12.70 0.000 -.7843553 -.57460343 _cons 1.965071 .455
24、0879 4.32 0.000 1.073115 2.857027 faminc .0108711 .0038504 2.82 0.005 .0033245 .0184177 grades -.2962217 .0526424 -5.63 0.000 -.3993989 -.19304462 1 (base outcome)psechoice Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-856.80718 Pseudo R2 =0.1589 Prob chi2 =0.0000 LR chi2(4)=323.70Multinomial
25、logistic regression Number of obs =1000.mlogit psechoice grades faminc ,baseoutcome(1)nologIIA assumption 3 -149.106 -147.165 3.880 3 0.275 for Ho 2 -158.961 -154.880 8.162 3 0.043 against Ho Omitted lnL(full)lnL(omit)chi2 df Pchi2 evidence Ho:Odds(Outcome-J vs Outcome-K)are independent of other alt
26、ernatives.*Small-Hsiao tests of IIA assumption(N=1000).mlogtest,smhsiaoThe randomnessExtensions have arisen to deal with this issue The multinomial probit and the mixed logit are alternative models for nominal outcomes that relax IIA,by allowing correlation among the errors(to reflect similarity amo
27、ng options)but these models often have issues and assumptions themselves IIA can also be relaxed by specifying a hierarchical model,ranking the choice alternatives.The most popular of these is called the McFaddens nested logit model,which allows correlation among some errors,but not all(e.g.Heiss 20
28、02)Generalized extreme value and multinomial probit models possess another property,the Invariant Proportion of Substitution(Steenburgh 2008),which itself also suggests similarly counterintuitive real-life individual choice behaviorThe multinomial probit has serious computational disadvantages too,s
29、ince it involves calculating multiple(one less than the number of categories)integrals.With integration by simulation this problem is being ameliorated now IIA assumption Slide16-21Principles of Econometrics,3rd Edition Total 1,000 100.00 3 527 52.70 100.00 2 251 25.10 47.30 1 222 22.20 22.20 colleg
30、e Freq.Percent Cum.=4-year college,3 2-year =1,2=no college .tab psechoicemlogit psechoice grades,baseoutcome(1)_cons 5.769876 .4043229 14.27 0.000 4.977417 6.562334 grades -.7061967 .0529246 -13.34 0.000 -.809927 -.60246643 _cons 2.506421 .4183848 5.99 0.000 1.686402 3.32644 grades -.3087889 .05228
31、49 -5.91 0.000 -.4112654 -.20631252 1 (base outcome)psechoice Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-875.31309 Pseudo R2 =0.1407 Prob chi2 =0.0000 LR chi2(2)=286.69Multinomial logistic regression Number of obs =1000Iteration 4:log likelihood=-875.31309 Iteration 3:log likelihood=-875.31
32、309 Iteration 2:log likelihood=-875.36084 Iteration 1:log likelihood=-881.68524 Iteration 0:log likelihood=-1018.6575 .mlogit psechoice grades,baseoutcome(1).tab psechoice,gen(coll)So we can run the individual logits by handhere“3-year college”versus“no college”_cons 2.483675 .4241442 5.86 0.000 1.6
33、52367 3.314982 grades -.3059161 .053113 -5.76 0.000 -.4100156 -.2018165 coll2 Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-308.37104 Pseudo R2 =0.0569 Prob chi2 =0.0000 LR chi2(1)=37.20Logistic regression Number of obs =473Iteration 3:log likelihood=-308.37104 Iteration 2:log likelihood=-308.
34、37104 Iteration 1:log likelihood=-308.40836 Iteration 0:log likelihood=-326.96905 .logit coll2 grades if psechoice|z|95%Conf.Interval Log likelihood=-328.76471 Pseudo R2 =0.2778 Prob chi2 =0.0000 LR chi2(1)=252.92Logistic regression Number of obs =749Iteration 5:log likelihood=-328.76471 Iteration 4
35、:log likelihood=-328.76471 Iteration 3:log likelihood=-328.76478 Iteration 2:log likelihood=-328.85866 Iteration 1:log likelihood=-337.82899 Iteration 0:log likelihood=-455.22643 .logit coll3 grades if psechoice!=2Coefficients should lookfamiliarBut check sample sizes!Slide16-25Principles of Econome
36、trics,3rd EditionSlide16-26Principles of Econometrics,3rd EditionSlide16-27Principles of Econometrics,3rd Edition*compute predictions and summarizepredict ProbNo ProbCC ProbCollsummarize ProbNo ProbCC ProbColl ProbColl 1000 .527 0 .527 .527 ProbCC 1000 .251 0 .251 .251 ProbNo 1000 .222 0 .222 .222 V
37、ariable Obs Mean Std.Dev.Min Max.summarize ProbNo ProbCC ProbColl(option pr assumed;predicted probabilities).predict ProbNo ProbCC ProbCollThis must always Happen,so do notUse sample values To assess predictive accuracy!Slide16-28Principles of Econometrics,3rd EditionCompute marginal effects,say for
38、 outcome 1(no college)grades 1000 6.53039 2.265855 1.74 12.33 Variable Obs Mean Std.Dev.Min Max.sum grades grades .0813688 .00595 13.68 0.000 .069707 .09303 6.53039 variable dy/dx Std.Err.z P|z|95%C.I.X =.17193474 y =Pr(psechoice=1)(predict,outcome(1)Marginal effects after mlogit.mfx,predict(outcome
39、(1)If not specified,calculation is done atmeansCompute marginal effects,say for outcome 1(no college)If specified,calculation is done atchosen level grades .0439846 .00357 12.31 0.000 .036984 .050985 5 variable dy/dx Std.Err.z P|z|95%C.I.X =.07691655 y =Pr(psechoice=1)(predict,outcome(1)Marginal eff
40、ects after mlogit.mfx,predict(outcome(1)at(grades=5)Another annotated examplehttp:/www.ats.ucla.edu/stat/Stata/output/stata_mlogit_output.htmThis example showcases also the use of the option rrr which yields the interpretation of the multinomial logistic regression in terms of relative risk ratiosIn
41、 general,the relative risk is a ratio of the probability of an event in the exposed group versus a non-exposed group.Used often in epidemiology In STATAmlogitNote that you should specify the base category or STATA will choose the most frequent oneIt is interesting to experiment with changing the bas
42、e categoryOr use listcoef to get more results automaticallyConsider testing whether two categories could be combinedIf none of the independent variables really explain the odds of choosing choice A versus B,you should merge themIn STATAmlogtest,combine(Wald test)Or mlogtest,lrcomb(LR test)mlogit pse
43、choice grades faminc ,baseoutcome(3)2-3 97.658 2 0.000 1-3 187.029 2 0.000 1-2 41.225 2 0.000 Alternatives tested chi2 df Pchi2 of alternatives are 0(i.e.,alternatives can be combined).Ho:All coefficients except intercepts associated with a given pair*Wald tests for combining alternatives(N=1000).ml
44、ogtest,combinemlogit psechoice grades faminc ,baseoutcome(3)Where does this come from?mlogit psechoice grades faminc ,baseoutcome(3)Prob chi2=0.0000 chi2(2)=187.03(2)1faminc=0(1)1grades=0.test1We test whether all the Coefficients are nullWhen comparing category 1 to the base,Which is 3 here 2-3 118.
45、271 2 0.000 1-3 294.004 2 0.000 1-2 46.360 2 0.000 Alternatives tested chi2 df Pchi2 of alternatives are 0(i.e.,alternatives can be collapsed).Ho:All coefficients except intercepts associated with a given pair*LR tests for combining alternatives(N=1000).mlogtest,lrcombmlogit psechoice grades faminc
46、,baseoutcome(3)These tests are based on comparing unrestricted versus constrained Regressions,where only the intercept is nonzero for the relevant category mlogit psechoice grades faminc ,baseoutcome(3)nolog est store unrestricted constraint define 27 1 mlogit psechoice grades faminc ,baseoutcome(3)
47、constraint(27)nolog est store restricted lrtest restricted unrestrictedYields:These tests are based on comparing unrestricted versus constrained Regressions,where only the intercept is nonzero for the relevant category:(Assumption:restricted nested in unrestricted)Prob chi2=0.0000Likelihood-ratio te
48、st LR chi2(2)=294.00Computational issues make the Multinomial Probit very rareLIMDEP seemed to be one of the few software packages that used to include a canned routine for itSTATA has now asmprobit Advantage:it does not need IIA Total 1,000 100.00 1 19 1.90 100.00 0 981 98.10 98.10 graduate Freq.Pe
49、rcent Cum.high school catholic =1 if .tab hscath _cons -3.642004 .6830122 -5.33 0.000 -4.980684 -2.303325 grades -.0471052 .1020326 -0.46 0.644 -.2470853 .1528749 hscath Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-94.014874 Pseudo R2 =0.0011 Prob chi2 =0.6445 LR chi2(1)=0.21Logistic regressi
50、on Number of obs =1000.logit hscath grades,nolog 1 (base outcome)_cons 3.642004 .6830122 5.33 0.000 2.303325 4.980684 grades .0471052 .1020326 0.46 0.644 -.1528749 .24708530 hscath Coef.Std.Err.z P|z|95%Conf.Interval Log likelihood=-94.014874 Pseudo R2 =0.0011 Prob chi2 =0.6445 LR chi2(1)=0.21Multin
