New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here


by: Golden Bernhard


Golden Bernhard
GPA 3.83


Almost Ready


These notes were just uploaded, and will be ready to view shortly.

Purchase these notes here, or revisit this page.

Either way, we'll remind you when they're ready :)

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

Class Notes
25 ?




Popular in Course

Popular in Statistics

This 21 page Class Notes was uploaded by Golden Bernhard on Friday September 18, 2015. The Class Notes belongs to STA 6127 at University of Florida taught by Staff in Fall. Since its upload, it has received 5 views. For similar materials see /class/206581/sta-6127-university-of-florida in Statistics at University of Florida.


Reviews for STAT METH SOC RES 2


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 09/18/15
Chapter 15 Logistic Regression Modeling Categorical Responses 688 CHAPTER 15 LOGISTIC REGRESSION The regression models studied in the past six chapters assume that the response variable is quantitative This chapter presents generalized linear models for response variables that are categorical Sections 151 7 153 present the logistic regression model for binary response variables 7 variables having only two possible outcomes For instance logistic regression can model 0 A voter s choice in a presidential election Democrat or Repub lican with predictor variables political ideology annual income education level and religious a liation 0 Whether a person uses illegal drugs yes or no with predictors education level whether employed religiosity marital status and annual income Multi category versions of logistic regression can handle ordinal re sponse variables Section 154 and nominal response variables Section 155 Section 156 introduces loglinear models which describe associ ation structure among a set of categorical response variables Section 157 shows how to check the goodness of t of models to contingency table data The models of this chapter use the odds ratio to summarize associations 151 Logistic Regression For a binary response variable 3 denote its two categories by 1 and O Commonly the generic terms success and failure are used for these two outcomes Recall from the discussion of Table 36 on page 61 and Example 48 on page 120 that the mean of O and 1 outcomes equals the proportion of outcomes that equal 1 Regression models for binary response variables describe the population proportions The population proportion of successes also represents the probability Py 1 for a ran domly selected subject This probability varies according to the values of the explanatory variables Models for binary data ordinarily assume a binomial distribution for the response variable Section 67 This is natural for binary outcomes The models are special cases of generalized linear models Section 144 Linear Probability Model For a single explanatory variable the simple model P 3 1 a M implies that the probability of success is a linear function of ac This is called the linear probability model 151 LOGISTIC REGRESSION 689 This model is simple but often inappropriate As Figure 151 shows it implies that probabilities fall below 0 or above 1 for suf ciently small or large x values whereas probabilities must fall between 0 and 1 The model may be valid over a restricted range of x values but it is rarely adequate when the model has several predictors Figure 151 Linear and Logistic Regression Models for a 0 1 Response for which ls Py 1 Fig 151 in Se replace pi on y aXis by Py1 The Logistic Regression Model for Binary Responses Figure 151 also shows more realistic response curves which have an S shape With these curves the probability of a success falls between 0 and 1 for all possible w values These curvilinear relationships are described by the formula 133 1 10g17 133 1 The ratio Py 11 7 Py 1 equals the odds a measure Section 84 introduced For instance when Py 1 075 the odds equals 075025 30 meaning that a success is three times as likely as a fail ure Software uses natural logarithms base 6 often denoted ln in tting the model However we won t need to use or understand logaritms to interpret the model and conduct inference using it This formula uses the log of the odds log Py 117 Py 1 called the logistic transformation or logit for short The model is abbreviated as 048w logitPy 1 04 890 It is called the logistic regression model When the logit follows this straight line model the probability Py 1 itself follows a curve such as in Figure 151 The parameter 8 indicates whether the curve goes up or goes down as w increases For 8 gt 0 Py 1 increases as w increases as in curve 1 in Figure 151 For 8 lt 0Py 1 decreases as w increases as in curve 2 in Figure 151 If 8 0 Py 1 does not change as w changes so the curve attens to a horizontal straight line The steepness of the curve increases as 8 increases For instance 8 for curve 2 is greater than 8 for curve When Py 1 050 the odds Py 117 Py 1 1 and logPy 11 7 Py O Equating this logit value of O to 04 853 and solving for 5c shows that Py 1 050 when w 7048 690 CHAPTER 15 LOGISTIC REGRESSION Most software uses maximum likelihood see Section 51 to t the model This method is more appropriate for binary data than least squares Example 151 Income and Having Travel Credit Cards Table 151 shows data for a sample of 100 adults randomly selected for an Italian study on the relation between annual income and having a travel credit card such as American Express or Diners Club At each level of annual income in thousands of euros the table indicates the number of subjects in the sample and the number of those having at least one travel credit card Let x annual income and y whether have a travel credit card 1 yes 0 no For instance for the ve observations at w 30 y 1 for two subjects and y 0 for three subjects Table 151 Annual Income in Thousands of Euros and Possessing a Travel Credit Card For example of the 5 subjects with income 30 thousand euros 2 possessed a travel credit card Number Credit Number Credit Number Credii Income Cases Cards Income Cases Cards Income Cases Cards 12 1 0 21 2 0 34 3 3 13 1 0 22 1 1 35 5 3 14 8 2 24 2 0 39 1 0 15 14 2 2 5 10 2 40 1 0 16 9 0 26 1 0 42 1 0 17 8 2 29 1 0 47 1 0 19 5 1 30 5 2 60 6 6 20 7 0 32 6 6 65 1 1 Source Thanks to R Piccarreta Bocconi Univ Milan The data were originally recorded in Italian lira but have been converted to euros Software provides results shown in Table 152 heavily edited The logistic prediction equation is logit3y 1 73518 010530 Since the estimate 0105 of 8 is positive the estimated probability of having a credit card increases at higher levels of income Figure 152 shows the prediction curve The estimated probability equals 050 at w 7646 35180105 335 The estimated probability of having a credit card is below 050 for incomes below 335 thousand euros and above 050 for incomes above this level 151 LOGISTIC REGRESSION 691 Table 152 Fit of Logistic Regression Model for Italian Credit Card Data B S E Exp B income 1054 0262 1111 Constant 3 5179 7103 Figure 152 Logistic Regression Prediction Curve for Example 151 Fig 153 in Se replace fr on y aXis by 1 and change values on w aXis to 12 24 36 46 60 72 Logistic Regression Equation for Probabilities An alternative equation for logistic regression expresses the probability of success directly It is 0z x PW 1 Here 6 raised to a power represents the antilog of that number using natural logs Most calculators have an 6 key that provides these an tilogs and software can report estimated probabilities based on the t of the model We use this formula to estimate values of Py 1 at particular predictor values From the estimates in Table 152 a person with annual income at thousand euros has estimated probability of having a credit card equal to 67362401055 133 1 1 73520105139 For subjects with income at 12 the lowest income level in this sample the estimated probability equals 673624010502 67226 0 133 1 1 7352010512 146426 1104 03909439 For x 65 the highest income level in this sample the estimated prob ability equals 097 692 CHAPTER 15 LOGISTIC REGRESSION Interpreting the Logistic Regression Model We ve seen how to estimate the probability of success and we ve seen that the sign of 8 tells us whether it is increasing or decreasing as w increases How else can we interpret 8 Unlike in the linear probability model 8 is not the slope for the change in Py 1 as w changes Since the curve for Py 1 is S shaped the rate at which the curve climbs or descends changes according to the value of w The simplest way to use 8 to interpret the steepness of the curve uses a straight line approximation to the logistic regression curve A straight line drawn tangent to the curve has slope Py 117Py 1 where Py 1 is the probability at that point Figure 153 illustrates The slope is greatest when Py 1 12 where it is 81212 84 So when Py 1 is near 12 onefourth of the 8 effect parameter in the logistic regression model is the approximate rate at which Py 1 changes per 1 unit increase in 3 Figure 153 A Line Drawn Tangent to a Logistic Regression Curve Has Slope Py 11 Py 1l Fig 152 in Se replace 7r by Py 1 For the Italian credit card and income data 0 0105 When the estimated probability of a credit card is 1 12 a line drawn tangent to the curve at that point has slope equal to 64 01054 0026 The rate of change in 133 1 for a 1 thousand euro increase in annual income is 64 0026 At the sample mean income of 2 25 the estimated probability of a credit card is 1 029 Then mm 117 133 1 0105029071 0022 so a 1 thousand euro increase in annual income relates approximately to a 0022 increase in the estimated probability of having a credit card Software can also t the linear probability model Py 1 04 653 The least squares t is 1 70159 001953 This formula also has about a 002 increase in 1 per thousand dollar increase in income However it provides quite different predictions at the low end and high end of the income scale For instance it provides the absurd prediction 1 gt 1 when w 2 84 Another way to describe the effect of w compares 1 at two different values of 3 We ve seen that when w increases from its smallest to its largest value in the sample Py 1 increases from 009 to 097 This is a strong effect as there is a very large change in 133 1 An 151 LOGISTIC REGRESSION 693 alternative is to instead evaluate 1 at values of x that are less affected by outliers such as the upper and lower quartiles Interpretation Using the Odds and Odds Ratio Another interpretion of the logistic regression parameter 6 uses the odds ratio measure of association Section 84 Applying antilogs to both sides of the logistic regression equation log Py 11 7 Py 04 x yields Py a x eoz gtc 1PU 39 The right hand side of this equation has the exponential regression form studied in Section 146 a constant multiplied by another constant raised to the ac power This exponential relationship implies that every unit increase in x has a multiplicative effect of e on the odds ln Example 151 the antilog of 6 is e eO39105 111 When annual income increases by 1 thousand euros the estimated odds of owning a credit card multiply by 111 that is they increase by 11 When an 25 for example the estimated odds of owning a travel credit card are 111 times what they are when w 24 When an 25 Estimated odds 11331 1gt e 339513103910m25gt 0414 PU whereas when w 26 mmmmmmT iiLiwmmeowq PU which is 111 times the value of 0414 at w 25 In other words eO39105 111 04600414 is an estimated odds ratio equaling the estimated odds at w 26 divided by the estimated odds at w 25 Odds ratios also apply to changes in or other than 1 For example a 10 unit change in w corresponds to a change of 106 in the log odds and a multiplicative effect of e106 e610 on the odds When an 30 the odds equal 11110 29 times the odds when w 20 Most software can report odds ratio estimates and estimated proba bilities Table 152 reports the estimated odds ratio for a 1 unit increase in or under the heading ExpB 694 CHAPTER 15 LOGISTIC REGRESSION 152 Multiple Logistic Regression Logistic regression can handle multiple predictors The multiple logistic regression model has the form logitPy 1 04 1w1 kkgt The formula for the probability itself is Oltl311ml3k rk PW 1 Exponentiating a beta parameter provides the multiplicative effect of that predictor on the odds controlling for the other variables The farther a 8 falls from 0 the stronger the effect of the predictor ccz in the sense that the odds ratio falls farther from 1 As in ordinary regression crossproduct terms allow interactions be tween pairs of explanatory variables Square terms allow probabilities to increase and then decrease or the reverse as a predictor increases To include categorical explanatory variables set up dummy variables as the next example illustrates Example 152 Death Penalty and Racial Predictors Table 153 is a threedimensional contingency table from a study1 of the effects of racial characteristics on whether individuals convicted of homicide receive the death penalty The variables in Table 153 are death penalty verdict the response variable having categories yes no and the explanatory variables race of defendant and race of victims each having categories white black The 674 subjects were defendants in indictments involving cases with multiple murders in Florida between 1976 and 1987 Table 153 Death Penalty Verdict by Defendant s Race and Victims Race for Cases with Multiple Murders in Florida Defendant s Victims Death Penalty Percent Race Race No es White White 53 414 113 Black 0 16 00 Black White 11 37 229 Black 4 139 28 1M L Radelet and G L Pierce Florida Law RB U LB UJ Vol 43 1991 pp 1734 152 MULTIPLE LOGISTIC REGRESSION 695 For each of the four combinations of defendant s race and Victims race Table 153 also lists the percentage of defendants who received the death penalty For white defendants the death penalty was imposed 113 of the time when the victims were white and 00 of the time when the victims were black a difference of 113 7 00 113 For black defendants the death penalty was imposed 229 7 28 201 more often when the victims were white than when the victims were black Thus controlling for defendant s race by keeping it xed the percentage of yes death penalty verdicts was considerably higher when the victims were white than when they were black Now consider the association between defendant s race and the death penalty verdict controlling for victims race When the victims were white the death penalty was imposed 229 7 112 117 more often when the defendant was black than when the defendant was white When the victims were black the death penalty was imposed 28 more often when the defendant was black than when the defendant was white In summary controlling for victims race black defendants were some what more likely than white defendants to receive the death penalty For y death penalty verdict let y 1 denote the yes verdict Since defendant s race and victims race each have two categories a single dummy variable can represent each Let d be a dummy variable for defendant s race and 1 a dummy variable for victims race where d 1 defendant white d 0 defendant black 1 1 victims white 1 0 victims black The logistic model with main effects for these predictors is logithQ 1ll 04 1d 2117 where 81 represents the effect of defendant s race controlling for victims race and 8 represents the effect of victims race controlling for defen dant s race Here e 31 is the odds ratio between the response variable and defendant s race controlling for victims race and e 32 is the odds ratio between the response and victims race controlling for defendant s race Table 154 shows output for the model t based on using the gener alized linear models option from the Analyze menu in SPSS The pre diction equation is logit5y 1 73596 7 08680 240471 Since d 1 for white defendants the negative coef cient of d means that the estimated odds of receiving the death penalty are lower for white defendants than for black defendants Since 1 1 for white victims the positive coef cient of 1 means that the estimated odds of receiving the 696 CHAPTER 15 LOGISTIC REGRESSION death penalty are higher when the Victims were white than when they were black Table 154 Parameter Estimates for Logistic Model for Death Penalty Data Esti mates equal 0 for the black category of each predictor because its dummy variable value is 0 B Std Error Exp B Intercept 3596 5069 027 defendantwhite 868 3671 420 defendantblack 0 Victimwhite 2404 6006 11 072 Victimblack 0 The formula for the estimated probability of the death penalty is 67359670868d2404v 133 1 1 6735967086801434041139 For instance when the defendant is black and the Victims were white at O and 7 1 so 67359670868024041 71192 03 4 0 133 1 1 67359670868024041 1 71192 023339 This is close to the sample proportion of 0229 Table 153 The es timated probabilities unlike sample proportions perfectly satisfy the model The closer the sample proportions fall to the estimated proba bilities the better the model ts The antilog of 11 namely equot1 e 039363 042 is the estimated odds ratio between defendant s race and the death penalty controlling for Victims race The estimated odds of the death penalty for a white defendant equal 042 times the estimated odds for a black defendant We list white before black in this interpretation because the dummy variable was set up with d 1 for white defendants If we instead let d 1 for black defendants rather than white then we get l 0868 instead of 70868 Then e03 238 which is 1042 that is the estimated odds of the death penalty for a black defendant equal 238 times the estimated odds for a white defendant controlling for Victims race For Victims race e2404 111 Since 7 1 for white Victims the estimated odds of the death penalty when the Victims were white equal 111 times the estimated odds when the Victims were black controlling for defendant s race This is a very strong effect 152 MULTIPLE LOGISTIC REGRESSION 697 This model assumes that both explanatory variables affect the re sponse variable but with a lack of interaction The effect of defendant s race on the death penalty verdict is the same for each victims race and the effect of victims race is the same for each defendant s race This means that the estimated odds ratio between each predictor and the re sponse takes the same value at each category of the other predictor For instance the estimated odds ratio of 111 between victims race and the death penalty is the same when the defendants were white as when the defendants were black D Effects on Odds The parameter estimates for the logistic regression model are linear ef fects but on the scale of the log of the odds It is easier to understand effects on the odds scale than the log odds scale The antilogs of the parameter estimates are multiplicative effects on the odds To illustrate for the data on the death penalty the prediction equa tion log5y 117 153 1 73596 7 0868d 240471 refers to the log odds ie logit The corresponding prediction equation for the odds is Odds 67359670868d240411 673596670868d624041 39 For white defendants d 1 and the estimated odds equal e 339596e 039363e239404 For black defendants d 0 and the estimated odds equal e 339596e239404 The estimated odds for white defendants divided by the estimated odds for black defendants equal e 039363 042 This shows why the antilog of the coef cient for d in the prediction equation is the estimated odds ratio between defendant s race and death penalty verdict for each xed victims race The effect of defendant s race being white is to multiply the estimated odds of a yes death penalty verdict by e 039363 042 com pared to its value for black defendants The actual values of the odds depend on victims race but the ratio of the odds is the same for each The logit model expression for the log odds is additive but taking antilogs yields a multiplicative expression for the odds In other words the antilogs of the parameters are multiplied to obtain odds We can use this expression to calculate odds estimates for any combination of defendant s race and victims race For instance when the defendant is black d 0 and the victims were white 1 1 the estimated odds of the death penalty are Odds 67359670868d240411 673696708680424040 71192 698 CHAPTER 15 LOGISTIC REGRESSION Since Py 1 odds1 odds the ratio 03041 0304 0233 is the estimated probability of the death penalty found above Example 153 Factors Affecting First Home Purchase Table 155 summarizes logistic regression results from a study2 of how family transitions relate to rst home purchase by young married households The response variable is whether the subject owns a home 1 yes 0 no The parameter Py 1 is the probability of home ownership Table 155 Results of Logistic Regression for the Probability of Home Ownership Variable Estimate Std Error Intercept 2870 7 Husband earnings 10000 0569 0088 Wife earnings 10000 0306 0140 N0 years married 0039 0042 Married in 2 years 1 yes 0224 0304 Working wife in 2 years 1 yes 0373 0283 No Children 0220 0101 Add child in 2 years 1 yes 0271 0140 Head s education no years 0027 0032 Parents home ownership 1 yes 0387 0176 The model contains several explanatory variables Two variables mea sure current income husband s earnings and wife s earnings Two vari ables measure marital duration the number of years the respondent has been married and marital status two years after the year of observation The latter is a categorical variable with levels married married with a working wife single single being the omitted category for the two dummy variables Two variables measure presence of children the num ber of children aged 0717 in the household and a dummy variable for whether the family has more children aged 0717 two years after the year of observation The other variables are the household head s education and a dummy variable for whether the subject s parents owned a home in the last year that the subject lived in the parental home In Table 155 the ratio of the estimate to its standard error exceeds 2 for four of the predictors Other things being xed the probability of home ownership increases with husband s earnings wife s earnings the number of children and parents home ownership For instance 2from J Henretta 506ml Forces Vol 66 1987 pp 52036 152 MULTIPLE LOGISTIC REGRESSION 699 each additional child had the effect of multiplying the estimated odds of owning a home b 6039220 125 that is the estimated odds increase by 25 A 10000 increase in earnings had the effect of multiplying the estimated odds of owning a home by 60569 177 if the earnings add to husband s income and by 60306 136 for wife s income From Table 155 number of years married marital status in two years and head s education show little evidence of an effect given the other variables in the model We could re t the model without these predictors This more parsimonious model may yield better estimates of the effects of the signi cant variables Effects on Probabilities We ve seen that we can summarize the effects of predictors by estimating odds ratios Many researchers nd it easier to get a feel for the effects by viewing summaries that use the probability scale Such summaries can report estimated probabilities at particular values of a predictor of interest This evaluation is done at xed values of the other predictors such as at their means or at certain values of interest To illustrate let s investigate the effect of husband s earnings for the above example Consider the estimated probability 1 of home ownership when wife s earnings 50000 years married 3 the wife is working in two years number of children 0 add child in two years 0 head s education 16 years and parents home ownership 0 When husband s earnings 20000 728700569203065700393 037317002716 133 1 1 728700569203065700393037317002716 04139 Then 1 increases to 055 if husband s earnings increase to 30000 to 079 if they increase to 50000 and to 098 if they increase to 100000 The effect seems quite strong Alternatively you can report the change in the estimated probability 133 1 when a predictor increases by a certain amount such as 1 by a xed value eg 1 2 by a standard deviation 3 over its range from its lowest to greatest value or 4 over its inter quartile range from the lower quartile to the upper quartile Approach 4 is unlike 1 not affected by the choice of scale and unlike 2 and 3 not affected by outliers 700 CHAPTER 15 LOGISTIC REGRESSION 153 Inference for Logistic Regression Models We next discuss statistical inference for logistic regression As usual inference assumes randomization for gathering the data It also assumes a binomial distribution for the response variable The model identi es y as having a binomial distribution and uses the logit link function for Py 1 which is the mean of y Wald and LikelihoodRatio Tests of Independence For the bivariate logistic regression model 10gitPy 1l a 806 H0 8 0 states that x has no effect on Py 1 This is the indepen dence hypothesis Except for very small samples we can test H 0 using a 2 test statistic dividing the maximum likelihood estimate 8 by its stan dard error Some software reports the square of this statistic called a Wald statistic This has a chi squared distribution with df 1 It has the same P value as the 2 statistic for the two sided H a 8 7 0 Most software can also report another test for this hypothesis Called the likelihoodratio test in general it is a way to compare two models a full model and a simpler model It tests that the extra parameters in the full model equal zero For example for bivariate logistic regression it tests H0 8 O by comparing the model logitPy 1 04 853 to the simpler model logitPy 1 04 The test uses a key ingredient of maximum likelihood inference the likelihood function Denoted by 6 this gives the probability of the observed data as a function of the parameter values The maximum likelihood estimates maximize this function Speci cally the estimates are the parameter values for which the observed data are most likely see Section 51 Let 0 denote the maximum of the likelihood function when H0 is true and let 1 denote the maximum without that assumption The formula for the likelihood ratio test statistic is e 72109 7210910 7 7210911 1 It compares the maximized values of 72 log 6 when H0 is true and when it need not be true There is a technical reason for using 72 times the log of this ratio namely that the test statistic then has approximately a chi squared distribution for large samples The olf value equals the number of parameters in the null hypothesis For testing H0 8 O with large samples the Wald test and likelihood ratio test usually provide similar results For small to moderate sample 153 INFERENCE FOR LOGISTIC REGRESSION MODELS 701 sizes the likelihood ratio statistic often tends to be larger and gives a more powerful test than the Wald statistic To be safe use it rather than the Wald statistic Example 154 Inference for Income and Travel Credit Cards Table 156 shows inference results for Example 151 page 690 about how the probability of having a travel credit card depends on income level H0 6 0 states that the probability of having a travel credit card is the same at all income levels Table 156 Logistic Regression Inference for Italian Credit Card Data B SE Wald df Sig 957o CI for expB income 1054 0262 16 24 1 000 1 056 1 170 Constant 35179 7103 2453 1 000 From the table 6 01054 has an estimated standard error of 00262 The 2 test statistic equals 2 0105400262 402 This has a P value of 0000 for Ha 6 7 0 Table 156 reports the square of this statistic the Wald statistic This chi squared statistic equals 162 with df 1 and has the same P value There is strong evidence of a positive association between income and having a travel credit card The likelihood ratio test of H0 6 0 compares the maximized value of 72 log 6 for this model logitPy 1 04 x to its maximized value for the simpler model with 6 0 In its Model Summary SPSS reports a 72 log value of 9723 for the model Compared to the model with only an intercept term SPSS reports eg if the model is built with the ForwardzLR method that the change in 72log would be 2659 with df 1 This is the likelihood ratio test statistic It also has a P value of 0000 The result of these tests is no surprise We would expect subjects with higher incomes to be more likely to have travel credit cards As usual con dence intervals are more informative We could obtain con dence intervals for 6 or for Py 1 at various values of w The formula for a con dence interval for Py 1 is beyond the scope of this text but is simple with some software eg SAS as the appendix shows A 95 con dence interval for the probability of having a credit card equals 004 020 at the lowest income level of 12 020 040 at the mean income level of 25 and 078 0996 at the highest income level of 65 D 702 CHAPTER 15 LOGISTIC REGRESSION Inference in Multiple Logistic Regression For testing the partial effect of a predictor in a multiple logistic regres sion model the parameter estimate divided by its standard error is a 2 test statistic The square of that the Wald statistic is a chi squared statistic with df 1 Most software also reports likelihood ratio tests which compare the 72log 6 values with and without the predictor in the model This is particularly useful if a categorical predictor has sev eral levels in which case it has several dummy variables and the test of its effect equates several parameters to 0 in H0 Example 155 Inference for Death Penalty and Racial Predic tors For the death penalty data Example 152 page 694 used the model logithQ 1gtl 04 1d 2117 with dummy variables 01 and 1 for defendant s race and victims race If 81 0 the death penalty verdict is independent of defendant s race controlling for victims race In that case the odds ratio between the death penalty verdict and defendant s race equals 60 1 for each victims race Software SPSS shows the results Std Wald B Error ChiSq Sig 957o CI for ExpB Intercept 3596 5069 5033 000 defendantwhite 868 3671 559 018 20 86 victimwhite 2404 6006 1603 000 341 3593 The defendant s race effect of 70868 has a standard error of 0367 The 2 test statistic for H0 81 0 is z 708680367 7236 and the Wald statistic equals 72362 559 shown in the table For the two sided alternative either statistic has a P value of 0018 Similarly the test of H0 8 0 has P 0000 and provides extremely strong evidence of a victims race effect The parameter estimates are also the basis of con dence intervals for odds ratios Since the estimates refer to log odds ratios after construct ing the interval for a j we take antilogs of the endpoints to form the interval for the odds ratio For instance since the estimated log odds ra tio of 2404 between victims race and the death penalty has a standard error of 0601 a 95 con dence interval for the true log odds ratio is 2404 i1960601 or 123358 From applying the antilog ie exponential function 6 to each end point the con dence interval for the odds ratio equals 61236368 154 LOGISTIC REGRESSION MODELS FOR ORDINAL VARIABLES 703 34 359 shown in the table For a given defendant s race when the victims were white the estimated odds of the death penalty are between 34 and 359 times the estimated odds when the victims were black Most software can also provide con dence intervals for probabilities 95 con dence intervals for the probability of the death penalty are 014 037 for black defendants with white victims 001 007 for black defendants with black victims 009 015 for white defendants with white victims and 0003 004 for white defendants with black victims D LikelihoodRatio Test Comparing Logistic Regression Mod els To compare a model with a set of predictors to a simpler model having fewer predictors the likelihood ratio test uses the difference in the values of 72log for the two models This is an approximate chi squared statistic with df given by the number of extra parameters in the full model This test is an analog of the F test for comparing complete and reduced regression models Section 116 To illustrate the model in Example 153 about home ownership page 698 has 72log 29312 After adding ve variables to the model relating to the housing market such as the median sale price of existing homes in the area 72log drops to 28411 The difference 29312 7 28411 901 is a chi squared statistic with df 5 since the more complex model has ve additional parameters This shows extremely strong evidence of a better t for the more complex model P lt 00001 So at least one of these variables provides an improvement in predictive power A comparison of the 72 log 6 values for a model and for the model containing only an intercept term tests the joint effects of all the predic tors This test is a chi squared analog for binary data of the F test in regression of H0 81 k 0 that none of the predictors have an effect on 3 154 Logistic Regression Models for Ordinal Variablesquot Many applications have a categorical response variable with more than two categories For instance the General Social Survey recently asked subjects whether government spending on the environment should in crease remain the same or decrease An extension of logistic regression can handle ordinal response variables 704 CHAPTER 15 LOGISTIC REGRESSION Cumulative Probabilities and their Logits Let 3 denote an ordinal response variable Let Py S 9 denote the probability that the response falls in category 9 or below ie in cate gory 1 2 or 9 This is called a cumulative probability With four categories for example the cumulative probabilities are 133 17Py S 2 133 1Py 27P3 S 3 133 1Py 2P3 3 and the nal cumulative probability uses the entire scale so 133 S 4 A c category response has 0 cumulative probabilities The order of forming the cumulative probabilities re ects the ordering of the response scale The probabilities satisfy Py 1gtSP3S2SWltP3SCgt139 The odds of response in category 9 or below is the ratio H3139 Pygt9 39 For instance when the odds equal 25 the probability of response in category 9 or below equals 25 times the probability of response above category 9 Each cumulative probability can convert to an odds A popular logistic model for an ordinal response uses logits of the cumulative probabilities With 0 4 for example the logits are 7 Py 1 7 1 my 3 1 10g lPlty gt1gtl 10 lm 2 Plty 3 Plty 4gtl logithQ S 2 log 10g logitP3 S 3 log log Since the nal cumulative probability necessarily equals 10 we exclude it from the model These logits of cumulative probabilities are called cumulative logits Each cumulative logit regards the response as binary by considering whether the response is at the low end or the high end of the scale where low and high have a different de nition for each cumulative logit 154 LOGISTIC REGRESSION MODELS FOR ORDINAL VARIABLES 705 Cumulative Logit Models for an Ordinal Response A model can simultaneously describe the effect of an explanatory variable on all the cumulative probabilities for 3 For each cumulative probabil ity the model looks like an ordinary logistic regression where the two outcomes are low category 9 or below77 and high above category 9 This model is logitPy j O ji j12ci 1 For c 4 for instance this single model describes three relationships the effect of w on the odds that y S 1 instead of y gt 1 the effect of w on the odds that y S 2 instead of y gt 2 and the effect of w on the odds that y S 3 instead of y gt 3 The model requires a separate inter cept parameter a for each cumulative probability Since the cumulative probabilities increase as 9 increases so do 04 Why is the model written with a minus sign before 8 This is not nec essary but it is how the model is parameterized by some software such as SPSS That way when 8 gt 0 when w is higher cumulative probabilities are lower But cumulative probabilities being lower means it is less likely to observe relatively low values and thus more likely to observe higher values of 3 So this parameterization accords with the usual formulation of a positive association in the sense that a positive 8 corresponds to a positive association higher w tending to occur with higher Statisti cal software for tting the model has no standard convention Software such as SAS that speci es the model as logitPy S a 8w will report the opposite sign for You should be careful to check how your software de nes the model so you interpret 8 properly The parameter of main interest 8 describes the effect of w on 3 When 8 0 each cumulative probability does not change as w changes and the variables are independent The effect of w increases as 8 in creases In this model 8 does not have a j subscript It has the same value for each cumulative logit In other words the model assumes that the effect of w is the same for each cumulative probability This cumula tive logit model with this common effect is often called the proportional odds model Figure 154 depicts the model for four response categories with a quantitative predictor The model implies a separate S shaped curve for each of the three cumulative probabilities For example the curve for P3 S 2 has the appearance of a logistic regression curve for a binary response with pair of outcomes 3 S 2 and y gt 2 At any xed so value the three curves have the same ordering as the cumulative probabilities the one for Py S 1 being lowest The size of 8 determines how quickly the curves climb or drop The common value for 8 means that the three response curves have 706 CHAPTER 15 LOGISTIC REGRESSION Figure 154 Depiction of Curves for Cumulative Probabilities in Cumulative Logit Model Fig 154 in 3e the same shape In Figure 154 the curve for 133 S 1 is the curve for Py S 2 moved to the right and the curve for Py S 3 moved even further to the right To describe the association we can form e as a multiplicative effect of a 1 unit increase in w on the odds for the cumulative probabilities For each 9 the odds that y S j multiply by e for each oneunit increase in at It is e rather than e because of the negative sign attached to 8 in the model formula Model tting treats the observations as independent from a multina mial distribution This is a generalization of the binomial distribution from two to multiple outcome categories Software estimates the pa rameters using all the cumulative probabilities at once This provides a single estimate for the effect of w rather than the three separate estimates we d get by tting the model separately for each cumulative probability If you reverse the order of categories of y ie listing from high to low instead of from low to high the model t is the same but the sign of reverses Example 156 Comparing Political Ideology of Democrats and Republicans Do Republicans tend to be more conservative than Democrats Table 157 for the subjects of age 18 30 in the 2004 General Social Survey relates political ideology to party a iliation We treat political ideology which has a ve point ordinal scale as the response variable In Chapter 12 on ANOVA methods we treated political ideology as quantitative by assigning scores to categories Now we treat it as ordinal by using a cumulative logit model Let x be a dummy variable for party a iliation with w 1 for Democrats and w O for Republicans Table 158 shows results of tting the cumulative logit model The table reports four intercept parameter estimates because the response variable political ideology has ve cat egories These estimates are not as relevant as the estimated effect of the explanatory variable party a iliation which is 72241 Since the dummy variable x is 1 for Democrats and since high values of 3 represent greater conservatism the negative value means that Democrats tend to be less conservative than Republicans Democrat are more likely than


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Bentley McCaw University of Florida

"I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

Janice Dongeun University of Washington

"I used the money I made selling my notes & study guides to pay for spring break in Olympia, Washington...which was Sweet!"

Jim McGreen Ohio University

"Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

Parker Thompson 500 Startups

"It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.