Popular in Department
verified elite notetaker
verified elite notetaker
verified elite notetaker
verified elite notetaker
verified elite notetaker
verified elite notetaker
This 30 page Bundle was uploaded by Samuel Croteau on Sunday May 22, 2016. The Bundle belongs to at Massachusetts College of Pharmacy & Health Sciences taught by in Spring 2016. Since its upload, it has received 18 views.
Reviews for stat bundle
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 05/22/16
Math 118: Chapter 11 Experiments and Observational Studies These class notes for Chapter 11 are intended to complement the material in the text. Please read pp. 291 – 308. Do pay particular attention to the section “What Can Go Wrong” on p. 305. Be sure to attempt the “Just Checking” questions on p. 302. The answers are on p. 314. In the last chapter we looked at sample surveys. In such studies we are interested primarily in describing—primarily through estimation—some aspect of the population. For example, describing what proportion of all registered nurses plan to retire in the next five years, or describing the average unemployment time in Massachusetts. By contrast, in Chapter 11we will be exploring studies in which the aim is not some much to describe as to establish a causal relationship between two variables. We will look at both randomized experiments and observational studies. In a randomized experiment the researcher assigns individuals to treatments at random, in order to observe and measure the response. By contrast, in an observational study the researcher cannot impose any treatments but must simply observe and measure the treatment and the response. In a well-designed experiment the researcher can actually establish a causal relationship between an explanatory and a response variable. Such a relationship is established by eliminating the effect of other, so-called confounding variables on the response. In an observational study it is impossible to eliminate the effect of confounding variables and so it is, strictly speaking, impossible to establish a causal relationship. Example 1 As part of a study of the development of social skills in young children a sociologist compared 30 three-year-old girls who have been cared for at home by a parent with 30 three-year-old girls who have been in day care from an early age. All the children lived in the same upper-middle class neighborhood just outside Chicago. The response variable was the score on a Test of Social Abilities (TSA) that was given to each child. The test measures a child's ability to interact appropriately with both adults and other children. Scores on the test can vary from 0 to 50 with higher scores indicating greater sociability. The mean score for each group appears below. Type of Care Mean TSA score Home care Y H = 27.7 Day care Y = 34.8 D Response Variable? Explanatory Variable? Experiment or Observational study? The mean score for the Day care girls is 7.7 points higher than that for the Home care girls. Can we conclude that attending day care rather than Home care increases sociability? Example 2 In the 1950's Johns Hopkins University investigators began a study of the physical and social benefits of low-cost public housing for the poor. The experimental group consisted of a random sample of 300 families from approximately 800 that had applied for a new public housing project in Baltimore and had been approved by the Baltimore Housing Authority. The control group consisted of 300 families selected at random from among the 500 that had applied for the same project but had been rejected by the Authority. All 600 families had lived in roughly the same area of the city. After three years the experimental group was found to be happier, healthier and more law-biding than the control group. Response Variables? Explanatory Variable? Experiment or Observational study? There was an association between type of housing and behavior. Can we conclude that living in public housing produces happier, healthier, and more law-biding people than other forms of housing? Example 3 Two hundred (200) male students in an introductory statistics course agreed to participate in the following experiment. The students were randomly assigned to one of four 'beer' groups, A, B, C, and D, each containing 50 students. Each of the 200 students took a series of tests that measured their average reaction time to simulated automobile incidents. Those students in group A then drank six bottles of non-alcoholic beer; those in group B were given two bottles of alcoholic beer and four bottles of non-alcoholic beer, those in group C, four bottles of alcoholic beer and two bottles of non- alcoholic beer, and those in group D, six bottles of alcoholic beer. (All the drinks were in identical bottles without labels.) Ten minutes after consuming the beers, all 200 students were given another series of tests to measure their reaction times. The results are summarized below. Group Number of Number of Mean increase in students alcoholic beers reaction time A 50 0 Y A = - .01 sec B 50 2 Y B = .28 sec C 50 4 Y C = .74 sec D 50 6 Y D = 1.28 sec Response Variables? Explanatory Variable? Experiment or Observational study? These data indicate a pretty strong association between the amount of alcoholic beer consumed and the increase in reaction time. Can we go one step further and conclude that the progressive increase in reaction time is caused by the amount of beer consumed? Example 4 In a prospective study of the effects of smoking on the incidence of lung cancer, 4,000 adult females living in rural Minnesota were identified and classified according to a rough measure of how many cigarettes they smoked per day (0, 10, 30, or 60). They were followed until death and for each group the fraction for whom the cause of death was lung cancer was computed. The results are given below. Group Number of cigarettes/day Number of Lung cancer women death rate A 0 1,000 pA = .004 B 10 1,000 pB = .039 C 30 1,000 pC = .108 D 60 1,000 pD = .239 Response Variables? Explanatory Variable? Experiment or Observational study? There is a clear association between the number of cigarettes smoked and the chance of dying from lung cancer. Do these data 'prove' that the more you smoke the greater your chance of dying from lung cancer? Example 5 In a 1994 study in the Minneapolis-St. Paul area approximately 800 healthy, working adults were recruited to participate in an experiment designed to measure the impact of influenza vaccine on various health outcomes including the incidence of upper respiratory illnesses. The subjects were randomly assigned to receive either influenza vaccine or placebo injections. One year after the injections the following results were obtained. Group Mean Number of Upper Respiratory Illness per 100 subjects Placebo 1.40 Vaccine 1.05 Response Variable? Explanatory Variable? Experiment or Observational study? There is a clear association between type of injection and the incidence of upper respiratory illness. Do these data 'prove' that an influenza vaccine significantly reduces the likelihood of getting upper respiratory illness? The Language of Experiments and Observational Studies 1. When we address the issue of cause and effect, we may ask the question: Does X cause Y? but what we really mean is "Do changes or differences in X cause changes or differences in Y". In Example 1, for instance, we ask "is the difference in mean sociability scores due to the difference in the type of care"? 2. The individuals in an experiment are called experimental units (EU). In agriculture the EUs may be plants. In biology they may be mice or fruit flies. We are most interested in situations where the EU s are humans, in which case we call them subjects. 3A. The variable, the change in whose behavior, we wish to study is called the response or the dependent variable and is usually designated Y. 3B. The variable which may "explain" or "cause" changes in the response variable is called the explanatory or the independent variable. It is usually designated X. Sometimes the variable X is called a factor. 4. The different values (if X is quantitative) or categories (if X is qualitative) of the explanatory variable are called treatments. The EU/subjects who receive the standard or customary treatment are called the control group. Those EU/subjects who receive the new or innovative treatment are referred to as the experimental group or groups. Make sure you can identify the subjects, the explanatory and the response variables, and the control and the experimental groups in the examples. Principles of Experimental Design Note: There is no unique set of experimental design principles and mine are a little different from those in the text; but, we come out at the same place in the end. Principle 1: Comparison When researchers seek to evaluate the effectiveness of a new ‘treatment’ whether a medical procedure, a pedagogical innovation, a new production arrangement in a manufacturing plant, or a new government program, it must be judged against a suitable standard or control. Example 6 The South End Community Health Center (SECHC) serves a generally low- income Latino population in Boston’s South End. Some years ago the center planned to experimentally introduce intensive nutritional counseling for all pregnant women attending the center. At first they planned to try out the scheme for a year on all eligible women. Then researchers associated with the center pointed out that it would be impossible to measure the effectiveness of the counseling without a control group who did not receive the counseling. So, then it was decided to randomly assign newly pregnant women to receive or not receive the counseling. Then, at the very last minute, a new director of the center refused to permit the randomization scheme to go forward, arguing that it was unethical to withhold such ‘clearly beneficial’ counseling from a pregnant woman. In the event, the center reverted to the original plan of giving the counseling to all eligible women. But then came the question; how to measure the effect of this nutritional counseling on mothers and their off-spring? (a) Use historical records? Possibly, but while the counseling was being introduced, other changes, other innovations were taking place in the center, so how can you pinpoint/distinguish the exact effect of the counseling? (b) Use a nearly health center? Possibly, but other centers have very different populations and so are not really suitable controls. Principle 2: Replication In this context, ‘replication’ refers not to the need to be able to repeat the study under the same conditions, but rather the need to have adequate numbers of subjects in each group. This is because subjects react differently, even to the same treatment. We need a large number of patients in each group in order to decide whether differences in the responses between the groups is due to chance or to a significant treatment effect. If we could be sure that subjects, given a specific treatment, would all respond in exactly the same way (as in the example below), then we would need only ???? subject(s) per group? Treatment A 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 21.6 Treatment B 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 24.7 In real life the responses are much messier and more ambiguous. Treatment A 27.9 7.1 19.1 18.4 22.7 21.3 21.8 31.6 25.5 42.1 14.7 13.6 20.9 23.8 27.3 21.9 19.3 25.8 29.7 15.5 13.8 29.6 15.0 22.7 14.8 14.7 20.7 29.0 21.2 16.6 Mean = 21.6 Treatment B 26.8 23.9 25.4 36.4 20.4 23.1 16.5 21.8 26.3 27.4 30.5 21.3 19.9 33.9 33.4 18.2 34.1 18.8 21.1 12.1 23.9 21.1 21.1 31.3 44.7 25.2 25.1 14.3 20.0 23.0 Mean = 24.7 There is substantial variability in responses within each group. Can we be sure that the difference in the group means (24.7 – 21.6 = 3.1) is caused by chance or by the fact that treatment B really does have a substantially higher mean response than does treatment A? Principle 3: Comparability To the extent that the two groups differ at baseline (ie. at the start of the study), there is always the danger that such difference could be responsible, in part at least, for the differences between the groups in their responses. Might the difference in results be due to the fact that one group was slightly older, slightly more healthy to begin with, or with a slightly greater fraction of males, than the other? Might the fact that one group of students scored higher than the other be due, not to the different teaching methods, but to the fact that one group was systematically smarter or more highly motivated than the other? In any study in which we wish to establish a causal relationship between two variables it is essential that the groups be comparable with regard to other possible confounding variables. But how can we guarantee such comparability? The 'ideal' experiment Give all the available subjects treatment A and follow them until the outcomes have been observed. At the same time, in a parallel universe: give the total group of subjects treatment B and follow them until the outcomes have been observed. So long as the type of treatment is the only difference in the two universes, any difference in outcomes must be an exact reflection of the treatment effect Example 6 Recently a study was undertaken to determine what effect, if any, aspirin has on blood-clotting time. Twelve adult females participated. The time (in seconds) that it took the blood from a pin-prick to clot was measured before and three hours after each was given two aspirin tablets. The results are given below. Subject Before Aspirin (B) After Aspirin (A) A - B 1 12.0 12.3 0.3 2 12.3 12.9 0.6 3 13.1 14.2 1.1 4 11.0 11.0 0 5 11.3 11.6 0.3 6 11.2 10.9 - 0.3 7 11.4 11.6 0.2 8 13.0 12.9 - 0.1 9 11.3 11.8 0.5 10 11.8 12.2 0.4 11 12.0 12.0 0 12 12.4 12.5 0.1 This is an example of a matched pairs design. In practice, six (randomly assigned) of the women will receive the aspirin first and the other six will receive the aspirin last. In studies where subjects cannot act as their own controls, subjects should be randomly assigned to the different treatments. With a large number of subjects, randomization will create treatment groups that are approximately comparable in the sense that there are no systematic differences between the treatment groups. Thus any difference in the response variable(s) can be attributed to the effect of the treatment (the explanatory variable). Even when random assignment is possible it is not always used - Example 2. On the next page is a table from the New England Journal of Medicine article which describes the flu vaccine experiment. The table shows base- line characteristics of the two groups (Placebo and Vaccine) after randomization. This table shows that, as baseline, the groups are approximately comparable with regard to these 14 or so potentially confounding variables. But the great power and beauty of randomization is that it equalizes the groups not just with regard to obvious variables but with regard to the zillions of variables we cannot even measure or are not even aware of. Randomization of subjects to treatment groups roughly equalizes the groups with respect to subject characteristics but there are other sources of bias unrelated to such characteristics. (a) Patient and doctor expectations - use double-blinded experiments. (b) When the experimental conditions differ. Random Assignment in Practice 1. Use the table of random digits in the back of the book Using Minitab for Random Assignment Suppose we want to divide 200 students at random into four groups each of size 50. 1. Assign a unique number 1, 2, 3, ...200 to each of the students, 2. Use the following Minitab procedure MTB > set c1 DATA> 1:200 DATA> end MTB > sample 200 c1 c2 MTB > print c2 2 60 179 34 57 56 190 117 89 69 Group A 29 164 159 84 170 168 66 115 63 108 58 83 146 104 81 78 154 21 175 130 99 173 20 128 196 119 103 47 184 4 118 135 90 37 123 46 148 32 192 88 27 64 199 151 169 74 156 127 23 143 Group B 1 161 48 110 139 70 6 198 3 171 65 11 54 8 129 73 116 163 147 189 191 120 122 183 82 101 155 150 105 174 14 114 51 194 10 106 30 157 162 152 140 195 126 61 87 136 111 35 7 80 Group C 9 98 15 125 79 107 31 44 124 166 26 86 92 181 25 96 160 33 77 153 45 133 132 52 100 41 67 131 68 76 121 138 91 200 59 188 149 145 95 24 134 17 19 53 102 43 55 12 22 197 Group D 193 18 185 40 71 142 176 75 137 141 28 165 158 39 182 113 36 112 167 172 50 42 177 62 13 109 178 72 5 97 85 16 186 38 93 187 94 144 49 180 Completely Randomized Design 200 student subjects Random Assignment A 50 B 50 C 50 D 50 0 beers 2 beers 4 beers 6 beers Compare increases in reaction times for the four groups Randomized Block Design Frequently it is important to achieve not just approximate, but exact comparability between the treatment groups. For example, suppose reaction time is sensitive to gender. Random assignment would make the gender distribution in the four groups roughly equal but this may not be good enough. To obtain exact comparability you need to set up blocks based on gender before randomizing. Suppose that 120 of the 200 students are female. 200 student subjects 120 females 80 males Random Assignment A B C D 30 F 20 M 30F 20M 30F 20M 30F 20M 0 beers 2 beer 4 beers 6 beers Compare increases in reaction time for the four groups Random Sampling/Selection and Random Assignment In surveys, random sampling/selection is vital in establishing the validity of the generalization from the results in the sample to the population. Random assignment in surveys is never an issue; we collect information from survey respondents but we don’t assignment to treatments. By contrast, in randomized experiments, random assignment is vital to establish a cause-and-effect relationship between the explanatory and the response variables. However, it is extremely unusual for subjects in an experiment to be randomly selected; subjects are invariably volunteers or are recommended to the researchers. As a consequence, in a well-designed randomized experiment we can infer a causal relationship between the explanatory variable (treatments) and the response variables (the outcome) but only for those subjects in the experiment. It is always difficult to generalize the results of randomized experiments. So, random selection is essential in surveys; random assignment is essential in experiments. Example 5 In a 1994 study in the Minneapolis-St. Paul area approximately 800 healthy, working adults were recruited to participate in an experiment designed to measure the impact of influenza vaccine on various health outcomes including the incidence of upper respiratory illnesses. The subjects were randomly assigned to receive either influenza vaccine or placebo injections. One year after the injections the following results were obtained. Group Sample Size Mean Number of Upper Respiratory Illness per 100 subjects Placebo 425 1.40 Vaccine 424 1.05 Difference 0.35 The large sample sizes and the the act of random assignment means that the we can assume that the difference in the mean number of URI (0.35 uri) is an exact reflection of the impact of the vaccine. Observational Studies Again Example 1 As part of a study of the development of social skills in young children a sociologist compared 30 three-year-old girls who have been cared for home by a parent with 30 three-year-old girls who have been in day care from an early age. Type of Care Mean TSA score Home care Y H = 27.7 Day care Y D = 34.8 Why can’t we say that it is the day care that is responsible for the increased level of sociability? Example 4 A prospective study of the effects of smoking on the incidence of lung cancer. Group Number of cigarettes/day Number of Lung cancer women death rate A 0 1,000 ˆ = .004 p A B 10 1,000 p B = .039 C 30 1,000 p C = .108 D 60 1,000 p D = .239 Why can’t we say that it is the increased smoking that is causing the increase in lung cancer? In observational studies, like experiments, the groups are defined by the values or the categories of a variable X, and we compare the groups on the basis of a response variable Y. But, in the absence of random assignment, we cannot claim that X causes Y, because we can never be sure that the groups are comparable. That is, there may exist variables, other than X, call them Z ,1Z ,2Z 3 ... on which the groups also differ. We say that the effect of X on Y is confounded with the effects of these variables on Y. These Z variables are called confounding variables. A confounding variable (Z) must satisfy the following conditions: 1. It must be associated with both the explanatory variable (X) and the response variable (Y) 2. The occurrence of Z must precede that of the explanatory variable (X). Example 1 Type of Mean TSA Soc. of Parents % Blue-eyed Mean # Care score Z 1 Z2 Siblings 3 Home care Y H = 27.7 Z1H = 36.2 67% 1.89 Day care Y D = 34.8 Z2D = 42.2 43% 1.82 Z1 Sociability of parents Z2Whether or not blue-eyed Z Number of siblings 3 Example 4 A prospective study of the effects of smoking on the incidence of lung cancer. Group Number of Lung cancer Occupation Mean cups Factor X? cig/day death rate Z coffee/day Z 1 3 (X) Exposed Z2 A 0 pA = .004 20% 1.92 12.8 B 10 pB = .039 19% 1.84 19.4 C 30 pC = .108 22% 1.86 31.2 D 60 pD = .239 17% 1.90 46.4 Z1 Whether or not exposed to carcinogens Z2Number of cups of coffee/day Z3Factor X Methods for Improving Comparability in Observational Studies When randomization is not possible the researcher does have some tools for eliminating the effect of some confounding variables. 1. Restrict the types of subjects For example, if you restrict your study to only white females between the ages of 55 and 60 then the variables race, gender, and age can be ruled out as confounding variables. What is the problem with restricting subjects in this way? 2. Match the groups on as many variables as possible In a longitudinal study of the impact of premature birth on school performance, approximately 350 premature infants were matched with 350 normal-birth children. They were matched with respect to gender, age of mother (in five year intervals), social class, and birth rank in family. The matching might have looked like this: Premature Infants Normal-birth Infants Match Gender Age Soc. Cl. Rank Gender Age Soc. Cl. Rank 1 Male 30-35 4 1 Male 30-35 4 1 2 Female 20-25 2 2 Female 20-25 2 2 3 Female 15-19 5 1 Female 15-19 5 1 : : : : : : : : : : : : : : : : : : : : : : : : : : : 350 Male 25-30 3 3 Male 25-30 3 3 Premature Infants Normal-birth Infants Mean scores on exam 12.4 14.2 given to 11-year-old Can we conclude that premature infants do less well in school because they are premature? In fact, the matching did not guarantee that the pairs of infants were equivalent on all confounding variables and there were systematic differences between the two groups in the degree of maternal care and interest in school progress. Matching tries to accomplish in observational studies what blocking does in randomized experiments. 3. A third method for removing the effect of confounding variables in observational studies is statistical adjustment using techniques such as multiple linear regression or multiple logistic regression (Math 229). © Robert Goldman, 2011-12 When an experiment is not possible the closest we can get to establishing a causal connection is if the association meets the following criteria: (a) The association is strong (b)The association is consistent (c) The alleged cause precedes the effect in time (d)The alleged cause is plausible (e) A mechanism linking the two variables can be established. These criteria were developed in the 1960’s in an attempt to link cigarette smoking to lung cancer, emphysema, and other health problems. Even if these criteria are all met (as they were in the case of smoking and lung cancer) the case for a causal relationship is compelling but not proven.
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'