INTRO TO RES IN EDUC
INTRO TO RES IN EDUC EDRM 700
Popular in Course
Popular in OTHER
Mrs. Dangelo Fahey
verified elite notetaker
This 31 page Class Notes was uploaded by Edwin Considine on Monday October 26, 2015. The Class Notes belongs to EDRM 700 at University of South Carolina - Columbia taught by Me Gredler in Fall. Since its upload, it has received 23 views. For similar materials see /class/229493/edrm-700-university-of-south-carolina-columbia in OTHER at University of South Carolina - Columbia.
Reviews for INTRO TO RES IN EDUC
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 10/26/15
EDRM 700 GREDLER Five Types of Research Experimental Research 7 One or more aspects of a situation are manipulated under carefully controlled conditions and preselected outcomes are measured Experimental studies therefore are intervention studies Example Does peer tutoring produce a greater understanding of science for sixthgrade students than smallgroup discussions The researcher selects a sample of subjects and divides the sample into the number of groups needed to test the hypotheses In the above example method of instruction is manipulated A peer tutoring method is assigned to onehalf of the students and small group discussion is assigned to the other half At the end of six weeks science achievement is measured using an appropriate instrument and the two groups are compared Other examples Does clientcentered therapy lead to higher satisfaction in adult clients than traditional therapy Does an incidentbased approach to discussing bullying raise the con dence level of thirdgrade children Key features I A planned intervention I Two or more groups compared on a quantitative outcome Single subject research 7 the study of the effects of an intervention on the behavior of one subject at a time typically implemented in situations in which group designs are not appropriate Observation is the typical method of data collection Example What are the effects of selfrecording one s actions on offtask behavior for a highschool student with an attentiondeficit hyperactivity disorder Does a reinforcement program for handraising reduce the incidents of screeching to get adult attention in an autistic eightyearold boy Key characteristics I A planned intervention I One research subject only I Measurement of a quantitative outcome Correlational Research 7 to determine whether and to what degree a relationship exists between two or more characteristics sometimes used to make predictions about certain behaviors Example How well do prior grades and student selfdirectedness predict science achievement in middle school The researcher selects a sample of subjects obtains quantitative measures of the targeted characteristics and calculates the relationship Other examples Is selfregulation related to math achievement in high school students Is there a relationship between a mindful approach to problem solving and selfregulation in middle school students Key features I NO INTERVENTION I One group only I At least two measurements Causal comparative research 7 also referred to as ex post facto afterthefact research 7 attempts to associate differences that already exist between or among groups of individuals with differences in a particular characteristic Unlike experimental research the researcher does not create the difference between or among the groups Instead the difference is a a preexisting characteristic that cannot be manipulated age gender or b a characteristic that cannot be manipulated ethically or easily teaching style Example Do lOyearolds of different gender differ in linguistic ability Two groups of subjects that differ on an existing characteristic are measured on some attitude skill or other characteristic to determine if there are differences Note Interpretations of causalcomparative research are limited the researcher cannot state that the identified factor is the cause 7 only that there is a difference between groups Other examples Do middleschool teachers behave differently to students of different genders Do boys who attended kindergarten score higher on social adjustment than boys who did not attend kindergarten Key features I NO INTERVENTION I A preexisting characteristic is the focus of the research I Two or more groups I Measurement of a quantitative outcome Descriptive 01 Survey Research 7 information is collected from a sample in order to describe some aspects or characteristics of the larger group to which the surveyed individuals belong Example What are the views of beginning teachers in District X about the mentoring system for new teachers The researcher selects a sample of subjects administers the survey and typically reports the percentages of respondents who respond differently to the questions Other examples How do parents in school district Y feel about the middleschool counseling program What are high school teachers views of the dimensions of effective school leadership Key features I NO INTERVENTION I A defined group I Administration of surveys or interviews I Typically 39 39 quot of r of r J on the answers EDRM 700 GREDLER SamplingExtemal Validity Researchers typically conduct studies on a sample of individuals The expectation is that the ndings will apply to the larger group of which the research participants sample are members The larger group is referred to as the population On occasion the entire population may be sampled for the research study Suppose for example the population is the senior class in a small high school The entire class of 60 students may be sampled for the study Typically however sampling the entire population is not feasible in terms of cost and logistics A sample from the population is studied instead Research problems typically identify the population of interest as indicated in the following examples Tenyearold boys and girls Do lOyearold boys and girls differ in linguistic ability Sixth graders taking science Does peer tutoring produce a greater understanding of science for 6Lh graders than smallgroup discussion University freshmen Do study habits in addition to other factors predict achievement in university freshmen Thirdyear teachers in What do thirdyear teachers in district X district X think of the district s mentoring program for new teachers Secondsemester freshmen What do secondsemester freshmen think of University 101 as an orientation to university life The population of interest referred to as the target population is often unavailable Examples are tenyearold boys and girls sixth graders taking science and university freshmen The population to which the researcher can generalize the study results is the accessible population The sample for the study is drawn from the accessible population Example Sixth graders taking science target population Sixth graders in Richland District I accessible population Ten percent of the sixth graders sample Because the researcher wants the results to apply to the population the sample should be as representative as possible of accessible population Stated another way the intent is for the results of the study to generalize to the population The extent to which the study results can be generalized to the population is an issue of external validity Random sampling methods Methods that have the greatest chance of producing a sample that is representative of the population are some type of random sampling The purpose is to ensure as much as possible that there is no systematic variation between the characteristics of the sample and the characteristics of the population In other words differences between the individuals selected for the sample and the individuals in the population are the result of only chance variation Simple random sample A simple random sample is one in which each and every member of the population has an equal and independent chance of being selected This means that only chance determines whether a person is or is not selected for the sample If the random sample is large enough it is likely to be representative of the population A simple way to illustrate the selection of a random sample is as follows a Put all the names of those in the population on slips of paper and place them into a large bowl b Then draw one paper slip from the bowl and record the name or number that it shows c Return that paper slip to the bowl and shake the bowl d Continue this process until the total number for the sample is reached The above process however is not feasible for even relatively small populations Therefore a random numbers table may used Several websites generate sets of random numbers eg wwwrandomizerorg Statistical software programs can select random samples The researcher first numbers the members of the population and enters the numbers into the computer so that a random sample can be generated Stratified random sampling The population of interest for the research study may not be homogeneous it may be composed of two or more subpopulations To ensure that these subpopulations are adequately represented in the sample the researcher may use stratified random sampling The steps in stratified random sampling are as follow a Identify the two or more subpopulations in the accessible population The subpopulations are referred to as strata b Randomly select research participants from each subpopulation The two methods of random selection from the subpopulations are equal allocation and proportional allocation Equal allocation In equal allocation equal numbers are selected from each subpopulation stratum For example if the three strata are Native Americans African Americans and European Caucasians then 13 of the sample would be randomly selected from each of these three groups Proportional allocation Suppose in the above example that Native Americans are only 5 of the population AfricanAmericans are 55 and European Caucasians are 40 If the equal allocation method were used Native Americans would be over represented in the sample Therefore the researcher would randomly select 5 of the individuals for the sample from Native Americans and 55 and 40 respectively from the other two groups Cluster sampling Simple random and stratified random sampling are methods for selecting individuals to participate in a research study However sometimes it is not feasible to select individuals for the study For example it is typically not feasible to pull a few students out of a classroom to participate in the research Also the focus of the study may be instructional methods or some other classroom process In these situations cluster sampling is used classrooms are the clusters Brie y cluster sampling is appropriate when a the cluster includes two or more members of the population and b each member of the population is contained in only one cluster Wiersma amp Jurs 2009 p 335 Cluster sampling differs from random sampling in that the random selection does not occur with individuals it occurs with clusters All the clusters are identified and the sample is chosen by randomly selecting clusters from the larger population of clusters For example suppose that the population of interest is 93911 grade students in algebra in the school district and the district has 60 9 grade classrooms A cluster sample of 20 of the 60 classrooms means that 12 classrooms will be randomly selected from the set of 60 Nonrandom sampling methods Two nonrandom sampling methods are systematic sampling and convenience sampling Systematic sampling In systematic sampling the selection of the first participant determines the rest of the sample For example if every 10 name in an alphabetical list of members of the population is to be selected that numerical distance determines subsequent selections The researcher can introduce randomness in the selection of the first participant for the sample Suppose that the researcher intends to select every 10Lh individual for the sample She puts the first ten names into a bowl and randomly selects one name This individual is then the first member of the sample Suppose that the name drawn is the 8Lh name on the alphabetical listing of names Selection of the remainder of the sample proceeds by choosing every 10 name beginning with the Sth name that was drawn randomly ie the 18m 281 38m 48Lh names and so on This process is referred to as systematic sampling with a random start Systematic sampling is often used in research that studies large populations and when lists of the population members are available Directors of institutional research often use this method because it requires less work than simple random sampling Researchers should be aware of one potential although unlikely problem with systematic sampling The problem occurs if there is a cyclical pattern of characteristics within the lists of population members For example suppose that a high school is planning to survey a sample of students on an important topic The homeroom teachers instead of sending the principal the alphabetical rolls of their homeroom students send lists that rank the homeroom students from high to low in achievement The principal selects every 10th student from each list of approximately 30 students beginning with the 2quotd student on each list From each list the selections are the 2 12m and 22quotd students Systematic sampling in this case results in a biased nonrepresentative sample because the poorest students those numbered 23 to 30 do not have a chance of being selected Convenience sampling At times obtaining a random or systematic sample is not possible and the researcher uses a convenience sample Basically a convenience sample is exactly what the name implies 7 the sample was conveniently available For example several grades may be available in a school because the school is implementing some new initiatives and the district is seeking some data about the initiatives In other cases a district professor or college may volunteer classes or courses for a study for various reasons In general a convenience sample should not be considered to meet the requirements for generalizing the findings of the study to a defined population The sample does not meet the requirements for assuming that the study has external validity That is the finding cannot be automatically extended to a defined population the population that the convenience sample represents is not clear When the researcher cannot avoid relying on a convenience sample two requirements should be met a extensive demographic information and other characteristics of the sample should be documented and b the study should be replicated with other samples to decrease the possibility that the research findings are only a onetime occurrence Fraenkel amp Wallen 2003 p 107 The issue of sampling bias A biased sample is one that does not represent the population A biased sample can be due to a number of sources Wiersma amp Jurs 2009 It is a threat when a nonrandom sampling is implemented or b random sampling occurs with a biased source p 33 1 For example the focus of the study may be activities for middleschool science but the convenience sample consists of only highability classes The findings are not generalizable to average or typical classes The research lacks external validity Perhaps the bestknown example of random sampling that resulted in a biased sample occurred in the 1936 presidential election Literary Digest conducted a survey using random sampling to identify the sample The survey results predicted that Alf Landon would win the presidential election not the incumbent Franklin D Roosevelt However telephone directories and automobile registration lists were used to select the samplei and these lists did not represent the voting population p 331 The issue of sampling bias is that the ndings from a biased sample cannot legitimately be generalized to the population because of either a systematic differences between the participants in the research study and the population that are known or b unknown differences that limit generalization to a population In such situations the research study lacks external validity References Fraenkel J R amp Wallen N E 2003 How to design and evaluate research in education 5Lh ed Boston McGrawHill Wiersma W amp Jurs S G 2009 Research methods in education Boston Allyn amp Bacon Homework questions Identify the type of sampling for each of the following examples and state a rationale for your choice 1 Ten percent of the 200 English teachers in a small district are selected by drawing names with replacement out of a bowl Student government determines student attitudes toward the campus cafeteria by interviewing the first 50 students crossing the quad at lunchtime A researcher is interested in the possible effects of music on learning A nearby school grants him permission to use two thirdgrade classrooms In one class soft violin and orchestra music is played every day for 6 weeks Music is not available in the other class A TV station is interested in the success of a new marketing strategy on heads of households in suburban neighborhoods A random sample of 11 suburban areas is selected Heads of household in each identified area are surveyed Four percent of the membership of American Educational Research Association AERA was selected to participate in a survey study The 12 divisions of AERA were identified and an equal number of members was randomly selected from each division From Mittog K C amp Thompson B 2000 A national survey of AERA members perceptions of statistical significance tests and other statistical issues Educational Researcher 294 1420 EDRM 700 GREDLER Threats to lnternal Validity in Qualitative Research In research studies both qualitative and quantitative researchers collect data according to the research plan They then interpret the data and make inferences about the ndings The concept of internal validity refers to the conduct ofthe study such that inferences from the data are accurate ie valid That is the study is conducted in such a way that extraneous factors on events are ruled out in the interpretation of the data In qualitative studies the researcher is the data collection instrument That is the researcher conducts 1 observations of individuals in naturalistic settings taking extensive notes about events and 2 indepth interviews of participants Two major potential threats to internal validity therefore are data collector characteristics and data collector bias For example gender age ethnicity language patterns or other characteristics may influence the nature of the data that is obtained ifthe characteristics are related to variables that are the focus of the study see page 182 in the text Data collector bias refers to the unconscious distortion of data during the data collection process See p 182 in the text Examples include phrasing questions different ways for different individuals and asking leading questions In studies in which interviews are the primary data source the researcher should standardize the procedures Included are 1 asking the questions of all participants in the same way 2 put the interviewees at ease by asking a general easytoanswer question rst 3 conduct all interviews in a location in which the individual feels comfortable and 4 ensure the interviewee that their particular answers cannot be identi ed with them In ethnographic studies in which the researcher spends an extended period of time in the setting he or she can become fatigued The result is failing to notice new or different events when they occur Another risk is going native taking the perspective ofthose who are in the particular setting to the extent that objectivity is lost Several methods for enhancing the credibility ofthe data should be implemented One general strategy suggested by Webb Campbell Schwartz and Sechrest 1966 and explicated by Denzin 1978 is that oftriangulation The process consists of four techniques used in data collection and analysis 1 Implement multiple methods for obtaining data eg interviews documents observations 2 Consult multiple data sources eg interview teachers students and staff 3 Use more than one researcherto collect the data 4 Rely on additional researchers for analyses Other strategies that contribute the credibility ofthe data are listed in Table 102 As indicated each serves an important function in the research process These strategies 1 serve as checks against narrow or distorted data collection and analysis and 2 enhance opportunities to obtain richly contextualized information Particularly important in checking the data are the testing of rival hypotheses and the search for negative cases Sadler 1981 in a discussion of the information processing limitations of humans cites research that indicates people tend to not process information that con icts with an already held hypothesis The issue is not that 3 people deliberately ignore discon rming instances Instead these instances are simply not perceived p 28 Thus the conscious search for rival hypotheses and negative instances is important Some discussions of triangulation give the impression that the use of multiple methods observers and sources serve to corroborate the data Instead they may tap into different aspects of a social phenomenon For example primary teachers in one project received cards containing brief math activities that were appropriate for transitional and noninstructional class time Although teachers reported extensive use ofthe activities observations of 200 classrooms indicated that only 14 activities of the total set were used Mathison 1988 p 16 The explanation is that the teachers were positive about use of the activities but would nd they had no unplanned time and did not want to reduce the time spent on other goals Thus implementation was difficult for them p 16 The importance of triangulation is that all of the outcomes whether convergent inconsistent and contradictory should be ltered through other knowledge about the setting Mathison 1988 In other words triangulation rarely provides a singular view of a situation but it can provide a rich and complex picture of the phenomenon under study Mathison 1988 p 15 References Denzin N 1978 Sociological methods New York McGraw Hill Goetz J amp LeCompte M 1984 Ethnography and qualitative design in educational research New York Academic Press Gredler M 1996 Program evaluation Columbus OH Merrill Lincoln Y amp Guba E 1987 But is it rigorous Trustworthiness and authenticity in naturalistic evaluation In W Shadish amp C Reichardt Eds Evaluation studies review annual Vol 12 pp 425436 Newbury Park CA Sage Mathison S 1988 Why triangulate Educational Researcher 172 1319 Patton M 1990 Qualitative evaluation and research methods Newbury Park CA Sage Sadler R 1981 lntuitive data processing as a potential source of bias in naturalistic evaluations Educational Evaluation and Policy Analysis 34 2531 Webb E J Campbell D T Schwartz R amp Sechrest L 1966 Unobtrusive measures Nonreactive research in the social sciences Chicago Rand McNally EDRM 700 GREDLER CausalInference Studies Causalinference studies test the proposition Alf p then q Kerlinger 1979 For example if teaching method A then an increase in achievement Two types of such studies are found in the literature They are experimental and quasiexperimental studies Experimental studies An experiment is a research study Ain which 1 one or more independent variables are manipulated and 2 the in uence of all or nearly all possible in uential variables not pertinent to the problem of the investigation is kept to a minimum Kerlinger 1979 p 111 Experiments reduce the in uence of extraneous variables in two ways 1 by randomly assigning subjects to treatment groups and randomly assigning treatments to the groups and 2 controlling the in uence of other variables particular to the study through careful selection of the design for the study and development of the procedures used in the study For example the researcher may control the in uence of subjects reading ability on achievement by further subdividing the treatment groups into high average and low reading ability this topic is discussed further in factorial designs The primary purpose of control is to rule out threats to valid inference in the study Cook amp Campbell 1979 Characteristics of experimental studies are l the use of at least one comparison 2 the randomization of subjects and conditions 3 the manipulation of variables and 4 the control of extraneous variables that can affect the dependent variable The strengths of experiments are l relatively high control of the experimental situation 2 the capability of manipulating variables alone or in concert with other variables 3 exibility in that varied aspects of theory can be tested and 4 the capability of being replicated Kerlinger 1979 Replication however does not refer to repetition or duplication Instead it refers to repeating a research study with variations such as different age of the subjects deletion or addition of a variable or change in locale A weakness of experiments can be lack of generalizability That is if the experiment is conducted in a laboratory situation the findings may not apply to the classroom or other real world settings This problem is referred to as ecological validity Quasiexperimental studies One of the limitations of research in field settings is that the researcher may be unable to assign individual subjects to treatments These situations are described by Stouffer 1950 and Campbell 1957 as quasiexperiments Comparisons in such studies Adepend on nonequivalent groups that differ from each other in many ways other than the presence of a treatment whose effects are being tested Cook amp Campbell 1979 p 6 The task facing the researcher is that of separating the effects of a treatment from those that result from the initial noncomparability between the average units in each treatment group p 6 Therefore the researcher must Aexplicate the specific threats to valid causal inference that random assignment rules out and then in some way deal with those threats p 6 However because quasiexperiments do not use randomization causal inference in these studies is m 39 39 quot Cook amp Campbell 1979 The two major types of quasiexperiments are nonequivalent group designs and interrupted timeseries designs An example of the former is the comparison of two or more intact school classes with each other and measures such as achievement taken before and after the treatment In interrupted timeseries designs treatment effects are inferred from comparing performance measures taken at many time intervals before and after a treatment Cook amp Campbell 1979 Introduction to Types of Designs Discussed in this section are oneway and factorial designs Oneway design Recall from the earlier discussion on experimental and quasiexperimental studies the notation for representing the basic designs R 0 X1 R 0 X2 0 X1 0 X2 0 O 00 True experimental design with random assignment to experimental and control groups Quasiexperimental design no random assignment These designs are not restricted to two groups The Solomon fourgroup design is an example If subjects are randomly assigned to the groups the notation is as follows Treatment R 0 groups R Control R 0 groups R X1 X1 X2 X2 0 O O 0 If subjects are not randomly assigned to groups the study is illustrated without the Rs Prior examples of experimental studies have discussed testing hypotheses about instructional methods For example a researcher may be comparing peer tutoring and cooperative learning in middleschool science Such a study also may include a control group that receives traditional instruction Assuming that intact classes are used the design is illustrated as follows X1 X2 0 X3 0 000 In this design classes are each randomly assigned to one of the three groups Each of the above designs is referred to as a oneway design In a oneway design only independent variable is manipulated In the above example the independent variable is method of instruction Oneway designs also may be illustrated as follows Example 1 Method of Instruction N 450 Peer Tutoring Cooperative Learning Control Group N 150 N 150 N 150 In this study peer tutoring cooperative learning and traditional instruction are referred to as levels of the independent variable Dependent variable scores on science achievement In the study a mean on science achievement is calculated for each group and the three means are compared using an inferential statistic Factorial designs The oneway design has limitations Many social settings are complex and designs that only can manipulate one independent variable are limiting Factorial designs however can assess the effects of more than one independent variable on an identified dependent variable A factorial design is one in which Atwo three or more independent variables are simultaneously used to study their independent and joint effects on a dependent variable Kerlinger 1979 p 94 In a factorial design 1 each independent variable has two or more levels and 2 all levels of each variable are investigated in combination with all levels of every other independent variable The proposition that is tested is Aif p then q given r and perhaps s conditions The advantages of factorial designs 1 are more sophisticated theory can be tested 2 more realistic problems can be investigated and 3 the joint in uence of variables can be studied Kerlinger 1979 p 74 Randomized block design If a selected independent variable operates in conjunction with one or more other variables in a situation a oneway design does not account for these variables AResearchers in the behavioral sciences expect that preexisting differences in subjects will be at least as important a predictor of their scores on the dependent variable as any treatment variable Maxwell amp Delaney 2000 p 354 Because the sensitivity of statistical tests in one s study is directly dependent on the size of the explained variance in the study incorporating relevant preexisting differences in the model for the study is important One of the researcher s options is to incorporate the relevant variable into the design of the study Another is to incorporate it into the analysis For example suppose a researcher is testing the effects of different types of lesson orientation on reading materials eg objectives advance organizer list of key topics The students39 reading abilities will interact in different ways within the treatment groups with the materials depending on reading levels within the groups Instead of a oneway design the researcher can include reading ability as a variable in a factorial design in which reading ability serves as a control or blocking variable In other words a factorial design includes two or more independent variables This study is illustrated as follows Example 2 Reading Lesson Organizers Advance Organizer Objectives Key Topics N 60 N 60 N 60 Reading High N 60 Ability Average N 60 Low N 60 Independent variables Reading lesson organizers 3 levels and reading ability 3 levels Dependent variable Science achievement This design assists in determining whether any of the lesson organizers is more effective with high average or low readers This design is referred to as a 3 x 3 factorial design 3 levels of reading lesson organizers and 3 levels of reading ability In this example reading ability is referred to as a control or Ablocking variable That is reading ability is a status variable it is a preexisting student characteristic that is used to increase the precision of the measurement of the dependent variable One approach to assigning subjects to treatments is to first rank them according to their scores on the blocking variable Then those with the highest scores are block one subjects with the next highest scores are block two and so on Maxwell amp Delaney 2000 p 396 The subjects in each block of the control variable are randomly assigned to treatments subject to the constraint of An subjects from that block placed in each treatment In this example students in the high reading level are randomly assigned to either the advance organizer objectives or key topics conditions Then students who are average are similarly randomly assigned as are students at the low level The resulting design is a randomized block design Three hypotheses are tested in this study One is the effect of reading lesson organizers columns on achievement The mean for columns 1 2 and 3 are compared to test this hypothesis See Example 3 Example 3 Type of Reading Organizer Objectives Advance Organizer Key Topics High Average Low 0Obj 0Adv Org 0Key T The second hypothesis is the effect of the different levels of reading ability rows on achievement The means for rows 1 2 and 3 are compared See Example 4 Example 4 Objectives Advance Org Key Topics Reading High OHA Ability Average 0AA Low OLA These two hypotheses are referred to as main effects The third hypothesis is the combined effect of type of lesson organizer and level of reading ability on achievement In other words every level of type of reading organizer objectives advance organizers key topics is crossed with every level of reading ability high average and low See Example 5 Example 5 Reading Ability Type of Rea ding Organizer Objectives Advance Organizer Key Topics High OObjHA OAdv OrgHA OKey THA Average OObjAA OAdv OrgAA OKey TAA LOW OObjLA OAdv OrgLA OKey TfLA Completely randomized design In the randomized block design in the prior example one of the independent variables is a status variable reading ability That is it cannot be manipulated Sometimes the researcher is interested in two variables that can be manipulated An example is a study to test types of review strategies and amount of review time Example 6 illustrates this design Example 6 Amount of Review Time Independent variables Review strategies 2 levels and amount of time 2 levels Review Strategies 1 2 Selfquestioning Verbal elaborations l 011 021 20 minutes 2 012 022 35 minutes Dependent variable Achievement Disign 2 x 2 factorial In example 6 subjects in the experiment should be randomly assigned to the four treatment combinations 1 Selfquestions20 minutes 2 Selfquestions35 minutes 3 Verbal elaborations 20 minutes 4 Verbal elaborations35 minutes When random assignment to the four combinations is used the design is referred to as a 39 39 y 39 39 39 factorial design The three hypotheses tested in this study are 1 the effects of review studies on achievement column means 2 the effects of amount of review time on achievement row means and 3 the effects of the interaction of types of review strategies and amount of review time on achievement means in the cells G11 G21 G12 and G22 In summary two major types of factorial designs are randomized block and completely randomized designs References Campbell D T 1957 Factors relevant to the validity of experiments in social settings Psychological Bulletin 54 297312 Cook T D amp Campbell D T 1979 Quasi 39 quot Design and analvsis issues for field settings Boston Houghton Mif in Jaccard J amp Becker M A 1990 Statistical for the behavioral sciences 2quotd ed Belmont CA Wadsworth Kerlinger F N 1979 B 39 39 39 research A 39 armroach New York Holt Rinehart amp Winston Maxwell S E amp Delaney H D 2000 Designing e neriments and 39 39 data A model comparison perspective Mahwah NJ Lawrence Erlbaum Associates Stouffer S A 1950 Some observations on study design American journal of Sociology 55 355361 EDRM 700 Gredler Ethics in Research Compared to research in the socalled hard sciences eg physics chemistry research in the social sciences eg education psychology sociology is relatively young Research on learning began in the late 19Lh century with simple tests of reaction times and documenting the learning of lists of nonsense words Two examples of unethical procedures in early psychological research For several years little thought was given to protecting human subjects from potential physical or psychological harm danger or discomfort that may result from experimental procedures The following two studies are not representative of the early research but they illustrate clear violations of ethical procedures The experiment with Little Albert In a nowfamous experiment reported in 1920 John Watson trained conditioned the fear reaction of an 18monthold child named Albert to respond to several soft furry objects Watson amp Rayner 1920 Albert was tested first to confirm that live animals and objects such as a human mask and cotton did not elicit his fear response Then for several trials occasions a white rat was presented to Albert and a laboratory assistant behind Albert struck a long steel bar with a hammer On the first pairing of the white rat and the loud noise the infant jumped violently on the second trial pairing of the white rat and sound he began to cry On the eighth trial the white rat alone elicited crying and crawling away Watson amp Rayner 1920 Five days later the fear reaction also appeared in response to a white rabbit Non furry objects such as the child s blocks did not elicit the fear response but mild fear reactions occurred in response to a dog and a sealskin fur coat The child s emotional response had transferred to furry animals and objects Watson had not obtained the permission of Albert s mother to conduct the experiment Also she removed him from the day care facility for poor working mothers before any remediation or Jquot 39 r A could be A J Today this experiment would not be allowed because the emotional trauma to the child is not outweighed by overpowering scientific concerns The M ilgram experiment A more recent study is the nowfamous obedience study conducted by Stanley Milgram 1963 In that study na39139ve research participants were deceived into believing that they were delivering everincreasing electric shock to a learner each time that person made a mistake on a learning task 1 Sample The study involved 40 males between the ages of 20 and 50 who responded to a newspaper ad or direct solicitation They were postal clerks high school teachers salesmen engineers and laborers Their educational level ranged from one who had not finished elementary school to those who held doctorates or other professional degrees The subjects were paid for their participation in the experiment p 373 Deceptive procedures The participants were deceived as to the purpose of the study and the role of the person who was the focus of the experiment Each experiment involved one na39139ve participant and one victim an accomplice of the experimenter The participant was told a false cover storyithat the purpose of the study was to determine the effect of punishment on learning Each participant also was told that he would participate as either a teacher or a learner Both the participant and the accomplice drew slips of paper from a hat to determine who would be the teacher and who would be the learner However the drawing was rigged so that the naive participant always was the teacher The teacher and learner were then taken into another room where the learner was strapped into an electric chair apparatus The experimenter explained that the straps were to prevent excessive movement while the learner received the shock p 373 The experimenter attached an electrode to the leamer s wrist along with electrode paste to avoid blisters and burns p 373 The experimenter also stated that Although the shocks can be very painful they cause no permanent tissue damage p 373 To add to the realism the research participant received a sample shock of 45 volts to the wrist from the shock generator The source was a 45volt battery wired into the shock generator The answers that the learner gave to the wordpair questions also were rigged There were approximately three wrong answers to one correct answer p 374 Equipment The shock generator had 30 switches all in a row and each switch was labeled clearly with the amount of voltage it theoretically delivered Amounts ranged from 15 to 450 Signs on the machine for groups of switches were labeled Slight Shock Moderate Shock and so on to Danger Severe Shock Also the two last switches were labeled XXX p 373 When a switch was depressed a light corresponding to the switch lit up in bright red an electric buzzing began an electric blue light labeled voltage energizer ashed the indicator 0 a voltage meter swung to the right and several clicks were heard Learning task The socalled learning task consisted of the following a a list of paired words that was read to the learner b the first word of each pair was then read aloud along with four choices c the learner indicated his choice of the correct answer the word originally th second term of the pair by pressing one of four switches in front of him p 373 The switch also lit up one of four quadrants in a box that sat on top of the shock generator Directions to the research participant The experimenter directed the research participant the teacher to read the list of paired words aloud then to read the first word of each pair and the four choices After the learner gave his answer the teacher was to deliver an electric shock increasing the voltage each time After reading through the list the teacher was to repeat the list over and over increasing the voltage at each error until the learner makes no mistakes Feedback from the learner The learner provides no feedback to the teacher until administration of the shock level or 300 volts At that point the leamer pounds on the wall of the room p 374 Then the learner provides no further answers 7 Experimenter feedback The experimenter informed the teacher that no response from the learner was to be treated as a wrong answer and the teacher must continue the shocks At any point at which the teacher was unwilling to continue his participation the experimenter responded with predetermined prods 1 Please go on 2 The experiment requires that you go on 3 It is absolutely essential that you go on 4 You have no other choice you must go on p 374 If the research participant refused to continue after prod 4 the experimenter terminated the experiment Debriefing No details of the debriefing are provided The researcher simply states that procedures were undertaken to assure that the subject would leave the laboratory in a state of well being A friendly reconciliation was arranged between the subject and the victim and an effort was made to reduce any tensions that arose as a result of the experiment p 374 Results 1 No research participant stopped before administering shock level 20 300 volts 2 A total of 14 participants stopped before the end of the experiment75 at the 300volt level 2 330 and 1 each at 345 360 and 375 volts Twentysix ofthe participants obeyed the experimenter to the end until the most powerful shock available The experimenter stopped the session at that point p 374 In the postexperimental interview the experimenter asked each participant How painful to the learner were the last few shocks that you administered to him Response option ranged from 1 not at all painful to 14 extremely painful The mean for the group was 1342 and the mode the response selected most often was 14 Legal Requirements for Research on Human Subjects In 1974 Congress enacted legislations to address the ethical treatment of participants in biomedical and behavioral research The legislation referred to as the National Research Act PL 9348 also is applied to social science research The Act established institutional review boards IRBs for research in institutions that receive federal funds for research Another provision of the Act established the National Commission for the Protection of Human subjects of Biomedical and Behavioral Research The Commission identi ed three basic principles of human interaction that are the foundation for ethical conduct in human research These principles are beneficence respect and justice Beneficence This principle refers to two responsibilities They are to maximize good outcomes and to avoid harm Sieber 1980 The three components in avoiding harm are a avoiding the use of harmful procedures b ensuring participant anonymity and c protecting the con dentiality of data records M aximization of good outcomes This component addresses the issue of social value The research should contribute to knowledge related to improving the human condition In the educational setting the purpose often is to identify the various principles practices and conditions that enhance learning The maximization of good outcomes also implies that members of an eligible population should not be denied remedial services simply for the purpose of research Avoiding harmful procedures Current ethical standards specify avoiding procedures that can generate negative physical psychological or social effects The code of ethics subscribed to by ethnographers who study subcultures such as an alternative school goes beyond simply avoiding harm to participants The code specifies that the researcher do no harm to either the people or the community under study Fetterman 1989 The ethnographer exercises care not to trample the feelings of community members and to not desecrate events objects rituals and traditions that the culture identifies as sacred p 120 Assurance of participant anonymity A major component of the 1974 Privacy Act is that identifiable information about individuals may not be disclosed outside the agency that collected the information without the prior consent of the individuals concerned The data become anonymous when the identifiers are removed anal the individual is included in a group sufficiently large for the data subject to get lost in the crowd Bing 1986 p 86 This requirement typically is not a problem in the quantitative measurement of outcomes Code numbers are assigned to individual scores and data typically are reported in the form of frequency distributions means and standard deviations Protecting the anonymity of individuals can be a problem in a qualitative study where losing the individual in the small qualitative crowd is more difficult Akeroyd 1991 p 98 One traditional mechanism is to use pseudonyms in references to individual participants However pseudonyms may not be sufficient to protect the identity of individuals when only one or two persons such as a school principal or guidance counselor are in a position to make particular observations Guba amp Lincoln 1981 p 278 In such situations the evaluator should seek to present the information in another way or to use other sources Fetterman 1989 p 134 Protection of con dentiality The term con dentiality refers to an explicit promise or contract to reveal no personal information provided by or about an individual except under conditions agreed to by the subject Maintaining the anonymity of participants is one aspect of con dentiality The researcher faces two other key responsibilities with regard to con dentiality One is the safeguarding of personal and con dential information from unauthorized access and disclosure This responsibility is of particular concern when computers are used to analyze qualitative data The data record may contain a great deal of information that is open and generally known some that is con dential and restricted and some that is known but not stated openly or acknowledged Akeroyd 1991 p 91 Any password used by the researcher to protect con dentiality of the information must be robust That is unauthorized individuals should not be able to guess the password The concept of privileged communication The second responsibility regarding con dentiality is that of encoding or aggregating information in the les so that it cannot be traced to particular individuals Researchers and participants in a study typically regard con dentiality as a contract to protect personal and private information However the accepted legal expression for con dentiality is privileged communication The privilege refers to a person s rights in a special relationship to prevent the disclosure in a legal proceeding of information provided in con dence in that relationship Fischer amp Sorenson 1991 At present legal statutes identify communications between physicians and patients clergy and penitents husbands and wives and attorneys and clients as privileged However privileged communication does not extend to researchers Therefore a promise of con dentiality to participants in a study is not suf cient to protect the information from disclosure in a court of law At Stanford for instance law enforcement of cials claimed access to the data from a questionnaire on drug use administered to high school students as a baseline for evaluating an intervention Cronbach amp associates Respect The principle of respect refers to the protection of the autonomy or freedom of persons and the wellbeing of nonautonomous persons e g children and prisoners Sieber 1980 The key to this protection is the process referred to as informed consent The individual is informed about the nature of the study and freely consents in writing to be a participant Informed consent is not necessary when test data that are a part of institutional records are used for a study and scores cannot be traced to individuals Informed consent is needed however when individuals are to be assigned to different groups or programs or there is some other manipulation in the educatioinal setting that requires their participation Justice The term justice when applied to classical research studies refers to the equitable treatment and equitable representation of subgroups within society Sieber 1980 p 54 In most human research the justice standard is operationally de ned as equitable subject selection The individuals who are studied should be those who will bene t from the research and the participants in a research project should not be overstudied p 54 Requirements of informed consent Informed consent involves three important characteristics First the consent must be based on knowledge Individuals should be informed of the general purpose of the study and any risks associated with their participation Second the individual must be competent to make legally biding decisions Unless specific circumstances have been identified adults who are 18 years old or older are considered to be legally competent Informed consent is sought from parents when the research involves minors Third the consent must be voluntary It must be obtained without coercion misrepresentation or undue inducement Also the individual must be free to discontinue participation at any time during the study 1981 exemptions to the National Research Act Guidelines for research submitted to Health and Human Services HHS were amended for riskfree projects Specifically certain projects which are riskfree for the participants are exempt from regulation by HHS However IRBs decide the studies that are exempt from HHS regulation The types of exempt projects are as follows 1 Research conducted in educational settings such as studies on the effectiveness of educational techniques curricula classroom management or instructional strategy research N Research that uses educational tests provided that the participants are anonymous types of tests cognitive diagnostic aptitude achievement 3 Survey or interview procedures except when all of the following conditions occur a Participants could be identified b The participant could be at risk on criminal or civil charges or hisher financial or occupational status jeopardized if the responses became public c Research involves sensitive aspects of participant behavior such as illegal behavior drug use sexual behavior or alcohol use 4 Observation of public behavior including observations by participants except when the three conditions in 3 apply 5 Collection or study of documents records existing data pathological specimens or diagnostic specimens if a these sources are available to the public or b the obtained information is from anonymous sources Fraenkel amp Wallen 2003 p 65 References Akeroyd A 1991 Personal information and qualitative research data Some practical and ethical problems arising from data processing legislation In R Lee amp N Fielding Eds Using computers for qualitative research Newbury Park Sage Bing J 1986 Beyond 1984 The law and information technology in tomorrow s society Data protection and social policy Information Age 82 8594 Cronbach L J and associates Toward reform of program evaluation San Francisco JosseyBass Fetterman D 1989 Ethnography step by step Newbury Park CA Sage Fraenkel J R amp Wallen N E 2003 How to design and evaluate research in education 5Lh ed Boston McGrawHill Fischer L amp Sorenson G P 1991 School law for counselors psychologists and social workers New York Longman Guba E amp Lincoln Y 1981 E quotective evaluation San Francisco JosseyBass Milgram S 1963 Behavioral study of obedience Journal ofAbnormal and Social Psychology 674 371378 Sieber J 1980 Being ethical professional and personal decisions in program Evaluation In R Perloff amp E Perloff Eds Values ethics and standards in Evaluation New Directions for Program Evaluation 7 5161 Watson J B amp Rayner R 1920 Conditioned emotional reactions Journal of Experimental Psychology 3 114 reprinted in American Psychologist 2000 555 313317 Homework questions Which of the following situations present ethical problems and why a b A researcher is investigating the shortterm effects of a highfat highsalt diet on the cholesterol of office workers The sample for the study is divided into two groups One group is required to follow the highfat highsalt diet for two months and the other group is not required to change their eating habits Cholesterol levels are measured before the study begins and at the end of two months Provisionally admitted students in two southeastern universities are the population of interest for a study The students must maintain a passing average in all classes for two semesters in order to gain full admission The study is investigating the effects of weekly twohour meetings with academic counselors A sample for the study is drawn by selecting every third name from the class rolls of provisionally admitted students in student orientation classes Onehalf of the sample receives the academic counseling and the other half receives no additional services Gradepoint averages GPAs are obtained at the end of the fall semester and compared for the two groups Students in Biology 101 are participants in a research study that is comparing the effectiveness of handson labs to virtual labs that are presented in computer software The informed consent form that the students sign states that the students are required to participate in the study d Students in the Biology 101 study are tested every two weeks in class The research assistant enters the scores in a computer file that lists the scores by class section Scores are entered for each student using the student s initials
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'