Psych 302 Study Final Exam Study Guide
Ch. 4 – Ethics:
A. Ethics in Science
1. Federal Policy
1. Ethics Documents
iii. Minimum Ethics Training Compliance:
1. CITI – training in human subjects and animal research
i. Scientists have an obligation to minimize bias in research
1. Biases Towards Hypotheses
a. Difficult given investment
2. Personal Biases:
a. Commitment to diversity
b. Need to consider taboo subjects and ideas formulated from
c. People with biases make decisions about what questions are
being studied, and shape what questions will be asked in
the future (scientists and funding agencies).
B. Nuremberg Code (1947)
i. Implemented after the Nuremberg Trials of Third Reich scientists for their experimentation on human test subjects
1. Mandate of voluntary consent of human test subjects
a. Excluded vulnerable populations
b. A participant must be allowed to stop at any time.
iii. Research Must:
1. Benefit humanity
2. Balance risks/benefits
3. Be well-designed and based off of previous research
C. Milgram Study (1960-1961)
a. Too what lengths will a person go if someone in authority tells them to do it? b. Human test subjects believed they were gibing harmful electrical shocks to other participants who were actually confederates If you want to learn more check out What is the difference between thinking and feeling?
D. Declaration of Helsinki (1964)
a. Ethical principles regarding medical research on human test subjects
b. Internationally Agreed Upon Code of Ethics
i. Developed by the World Medical Association
ii. Foundation and basis for future codes of ethics and laws across other nations
c. Further Developed Nuremberg Code
i. Protection and inclusion of vulnerable groups
ii. Physicians moral obligation to look after his or her subjects is absolute iii. Research Ethics Committee must approve – foundation of IRBs If you want to learn more check out What is the definition of the discrete trial method experiment?
E. Tuskegee Syphilis Study (1932-1972):
i. Longitudinal Study conducted by the Public Health Services.
ii. 200 Uninfected black men and 300 infected black men.
iii. In 1943 Penicillin became the SOC for treating Syphilis but the men still were not given treatment, because as another goal of the experiment was to see the long-term effects of the disease in post-mortem patients.
b. Three Kinds of Ethics Violations:
i. The men were not treated respectfully
1. The researchers lied to them about what the experiment was
2. Did not allow the participants to give their full consent.
3. The low-income families may have felt that they were coerced.
ii. The men in the study were harmed.
1. The men were not given or told about the treatment that could have cured their disease.
iii. The researchers targeted a disadvantaged social group.
c. Direct Consequences:
i. 1974 National Research Act: We also discuss several other topics like What is the meaning of hearing in psychoacoustics?
1. Established the rule that all federally funded research involving
human test subjects must be approved by an IRB.
ii. 1979 Belmont Report:
1. Identified three core principles for research involving human test
F. Belmont Report
a. Respect for Persons:
i. Informed Consent:
1. Individuals participating in research should be able to make their Don't forget about the age old question of What are the negative ramifications of the problem?
own decisions and every participant is entitled to the precaution -
they should know all the costs and benefits. Don't forget about the age old question of What is the meaning of the state of nature?
2. No Coercion or Undue Influence:
a. Coercion occurs when researchers explicitly suggest that
the participant will suffer a negative consequence for not
doing what they say
b. Undue Influence occurs when researchers offer an Don't forget about the age old question of What are some of the biological and psychological factors that contribute to aggression?
incentive to attractive to refuse
1. Everyone is entitled to autonomous actions and some groups
require special protection to insure it.
2. Ex: (Prisoners, children, people with disabilities)
3. Information must be easily understood
i. Do no Harm
1. Researchers must take precautions to protect participants from
harm and to insure their well being.
2. Harm can be both psychological and physical.
ii. Maximize Benefits and Minimize Risks
1. Consider who benefits from the researcher and who might be
1. Fair match between who bears the burden and who could receive
2. Protection of vulnerable populations
3. Inclusion of Relevant Populations
a. Researchers might first ensure that the participants
involved in a study a re representative of the kinds of
people who would also benefit from its results.
G. Ethical Principles
a. Five General Ethical Principles
1. Do no harm
1. Fair and balanced
iii. Respect for Persons
1. Informed consent
1. Having moral principles
a. Ex: Professors are obligated to teach you accurately and
therapists are required to say up-to-date on empirical
evidence of therapeutic techniques
v. Fidelity and Responsibility:
a. Consistency and commitment to professional behavior.
Take ownership of ethical behavior
b. A psychologist must remain dedicated to his or her patients
and must execute to the best of their ability promises that
this relationship entails. This relationship must be kept
professional to the role it serves.
c. Ex: A clinical psychologists who teaches in a university
may not serve as a therapist to one of his or her classroom
d. Ex: Psychologists must avoid sexual relationships with
their students or client
b. Ten Specific Ethical Standards
1. Psychologist members of the APA who violate any of these
standards can lose their professional license or may be disciplined
in some other way by the association.
2. Ethical Standard 8: Research and Publication is by far the most
important for the context of this class
H. Ethical Standard 8:
a. International Review Boards (8.01)
i. A committee responsible for the interpretation of ethical principles and who ensure that research using human and animal participants is
ii. Before conducting a study, researchers must fill out a detailed application describing their study.
iii. If an institution conducts research using federal money (such as grants from the government), then a designated IRB is required.
iv. An IRB panel in the U.S. includes at least five people from specific backgrounds
1. >1 Scientist
2. >1 Person with academic interests outside the sciences
3. >1 A community member who has no ties to institution
4. Research for prison populations must include a member who is a
5. Research involving children will be heavily scrutinized.
b. Informed Consent (8.02)
i. Researchers are obligated to explain the study to potential participants in everyday language. Usually must have written documentation of
procedures, cost/benefit, what is being recorded, privacy etc.
ii. Written Consent
1. Indicates informed and voluntary participation
2. Two copies, one for the researcher and one for the participant
3. Written consent might not be needed when participants answer
completely anonymous questionnaires, and forms may not be
required for naturalistic observations
iii. When Consent is not Possible
1. Waiver documentation of consent
2. Assent documentation with children
c. Deception (8.07)
1. Should only be done when there are no other alternatives
2. There is no foreseeable harm to the participants
3. The research must be important
ii. Researchers withhold some details of the study from the participant. 1. Omission: Not informing participants of the full scope of the
2. Commission: Actively lying to the participants
d. Debriefing (8.08)
i. When researchers use deception, they must spend time debriefing each participant in a structured conversation.
1. Must be done as soon as possible
2. Must remove harmful effects of deception
3. Cost/Benefit analysis must be done here. Debriefing should not be done if it would cause more harm than good
ii. Researchers describe the nature of the deception and explain why it is necessary.
1. Must emphasize the importance of their research and must attempt to have honest relationship with the participant.
2. Non-deceptive studies often include a debriefing session as well e. Animal Research (8.09)
1. Must follow APA standards and AWA.
2. Research must have IACUC (Institutional Animal Care and Use Committee)
ii. Animal Care Guidelines:
a. Researchers should find alternatives to animals in
researcher when necessary.
a. Researchers must modify experimental procedures and
other aspects of animal care to minimize or eliminate
a. Researchers should adopt experimental designs and
procedures that require the fewest animal subjects possible
1. Resulted in numerous benefits
2. Animal research has contributed to countless valuable lessons about biology, psychology, neuroscience
3. Has made fundamental contributions to both basic and applied science
f. Research Misconduct
i. Data Fabrication/Falsification (8.10):
a. Occurs when, instead of recording what really happened in
a study, researchers invent data that fit their hypotheses.
a. Occurs when researchers influence the study’s results,
perhaps by selecting deleting observations from a data set
or by influencing their research subjects to act in the
a. Scientists use data to test their theories, and they can do so
only if they know that previously reported data is true and
a. In many universities, the reputation, income, and
promotions of professors are based off their publication and
their influence in the field.
ii. Plagiarism (8.11):
1. Representing the ideas or words of others as one’s own without
giving proper credit.
2. To avoid, a writer must cite the sources of all ideas that are not his
or her own, to give appropriate credit to the original authors.
Ch. 12 - Factorial Designs:
a. Factorial Designs (Factor = IV/predictor variable):
i. Use two or more independent variables/predictors that are
presented/manipulated at the same time.
ii. In the most common factorial design, researchers cross the two
independent variables to study each possible combination, creating a
condition representing each possible combination of the two (overlaid two independent variables)
1. This process creates all the number of condition types or cells.
iii. Allow researchers to test for:
1. Interaction Effects:
a. Whether the effect of the original independent variables
depends on the level of another independent variable
b. Interactions can allow researchers to ask, “Does cell phone
effects on driving depend on age?”
2. Multiple Hypotheses at once
a. All variables have two levels
b. Intuitive Interactions:
Much of the most important research in psychology explores interactions among multiple independent variables.
i. Crossover Interactions:
1. Lines measuring the effect of the interaction of two independent
variables on a dependent variable in which both IVs cross over one
2. Difference in Differences: Has two measurements one of negative
value and one of positive value across the interception
ii. Spreading Interaction:
1. Lines are not parallel and they do not cross over each other.
2. When there is an interaction, one can describe it accurately from
a. It is equally accurate to say: “When I am not holding a treat
there is zero difference between the ‘say sit’ and ‘say
nothing’ conditions; or “When I am holding a treat, there is
a large difference between the ‘say sit’ and ‘say nothing’
b. Often times one can see an interaction
i. Can be manipulated (true IVs), measured (participant variables), or a combination of the two (mixed)
1. Participant Variable:
a. A variable whose levels are selected (measured) not
manipulated. Because the levels are not manipulated,
variables such as age, gender, and ethnicity are not truly
independent variables. However, they are used as such in
B. Testing Limits
a. One purpose of factorial designs is to test whether an independent variable affects different kinds of people, or people in different situations in the same way. b. External Validity:
i. When researchers test an independent variable in more than one group at once, they are testing whether the effect generalizes.
c. Showing moderators
i. The process of using a factorial design to test limits is sometimes called testing for moderators (a variable that changes the relationship between two other variable)
1. An independent variable that changes the relationship between
another independent variable and a dependent variable
2. Results in an interaction the effect of one independent variable
depends on the level of another independent variable
C. Testing Theories:
a. In a factorial design, researchers test each independent variable to look for a main effect – the overall effect of one independent variable on the dependent variable, averaging over the levels of the other independent variable.
b. Main effect is a simple difference. In a factorial design with two independent variables, there are two main effects.
c. Marginal Means:
i. The arithmetic means for each level of an independent variable, averaging over levels of the other independent variable. If the sample size in each cell is exactly equal, marginal means are simple average. If the sample means are unequal, the marginal means will be computed using the
weighted average, counting the larger sample more.
ii. Main effects may or may not be statistically significant.
iii. Main effect = overall effect
1. When a study’s results show an interaction, the interaction itself is the most important effect
2. The overall effect of one independent variable at a time
D. Detecting Interactions
a. Interactions from a Table
i. A table can be used to determine if a study’s results show an interaction ii. Start by computing the two differences (an interaction), then go to the second level of the first independent variable (always compute the
differences in the same direction)
b. Interactions from a Graph
i. On a plotted line graph, see whether or not the lines are parallel. If not, there is probably an interaction
ii. On a bar graph, imagine connecting the tops of each matching bar with straight lines. If the lines are close, then you would assume that the graph is parallel and there wouldn’t be an interaction
c. Describing Interactions in Words
i. One of the simpler verbal descriptions starts with one level of the first independent variable, explaining what is happening with the second
independent variable at the same level, then moves to the next level of the first independent variable etc.
1. Ex: “When people saw photos of alcohol, they were quicker to
recognize aggression words than neutral words, but when people
saw pictures of plants, they were slower to recognize aggression
words over neutral words”
d. Interactions are almost always more important than main effects
E. Factorial Variations
a. Independent-Groups Factorial Designs
i. In an independent-groups factorial design (between-subjects factorial) both independent variables are studied as independent groups
1. For a 2X2 Factorial design there will be four different groups
(cells) of participants in the experiment
2. Some light men drank the placebo alcohol, some light men drank
the real alcohol, some heavy men drank the placebo alcohol, some
heavy men drank the real alcohol
b. Within-Groups Factorial Designs
i. (Repeated-measures factorial) both independent variables are manipulated within-groups
ii. There is only one group of participants but for a 2X2 design, they would participate in all four cells
1. All participant saw both alcohol photos and plant photos which
alternated over trials, all participants also responded to both
aggression-related words and neutral words
c. Mixed Factorial Designs
i. One independent variable is manipulated as an independent groups and the other ins manipulated as within groups
1. Ex: Age was an independent groups participant variable as
participants are either young or old, but the cell phone condition
was a manipulated independent variable as both young and old
participants were exposed to it.
F. Increasing the Number of Levels of an Independent Variables
a. General Guidelines
i. The quantity of numbers indicate the numbers of independent variables ii. The value of each of the numbers indicate the number of levels for each of the independent variables
iii. The product of each of the numbers is the number of cells
iv. Marginal means can still be computed, easiest way to detect a interaction is through computing the marginal means and plotting them on a graph G. Increasing the Number of Independent Variables
a. General Guidelines
i. Best way to depict a three-way design is to construct the original 2X2 Table, twice and create side by side graphs
ii. Three independent variables will result in three main effects to test, each representing a simple, overall difference: the effect of one independent variable, averaged across the other two independent variables
1. (The independent variable at the bottom of the chart is the one in
which the main effect is being tested for)
iii. When describing each main effect, you don’t mention the other two independent variables because you averaged across them
1. A three-way interaction, if it is significant, means that the two-way
interaction between two of the IV depends on the level of the third
iv. Three-Way Interaction Present If:
1. You would find a three-way interaction whenever there is a two
way interaction for one level of a third independent variable but
not for the other.
2. If a graph shows two different two-way interactions
H. Statistical Test
i. Output F statistic and p value
ii. Effect Size
1. Eta squared (n)
2. Partial eta squared (np^2)
a. Values are smaller than for “d” and “r” for even strong
b. .2 is a strong effect size
I. Identifying Factorial Designs in Readings
a. In Journal Articles
i. Look for:
1. Dual task
b. In Popular Press
i. Look for:
1. It depends
2. Look for participant variables
Ch. 13 – Quasi-Experiments and Small–N Designs:
i. Researchers do not have full experimental control.
ii. Start by selecting an IV and DV, then participants are exposed to each level of the IV
iii. Researchers might not be able to randomly assign participants to one level or the other
iv. Similar to correlational studies in threats to internal validity, however, more manipulation of variables is done in Quasi
i. Ecologically valid/real life groups or changes
ii. Contains Internal validity threats
iii. The best design is the one that answers a research question the best
c. Contrasting Designs:
i. True Experiment:
1. A study or research design in which the variables are manipulated.
Experimenter has complete control over the assignment of subjects
to conditions and presentation of conditions to participants
1. Experimenter has no control over presentation of IVs; can only
occur what is happening
1. Researchers select participants for different conditions from pre
a. “Ex post facto” experiments – “after the fact”
2. No random assignment but experimenters still have some
manipulation and experimental control
3. Always a trade off between ecological validity and internal
a. Preexisting groups differences
b. Confounds (subject variable) exist when participants are
assigned to conditions based upon group membership
B. Group Designs
a. Repeated-Measures Design
i. Participants experience all levels of an independent variable. The researcher takes advantage of an already-scheduled event, a new policy or regulation, or a chance occurrence to manipulate the independent variable. b. Interrupted Time-Series Design:
i. A quasi-experimental study that measures participants repeatedly on a dependent variable before, during, and after the “interruption” caused by some event
ii. Extension of the pretest/posttest design as it allows the same group to be compared over time by considering the trend data before and after the experimental manipulation
1. Ex: Judicial decision making: As the day went on, judges were increasingly less likely to approve parole, unless the hearing came directly after a snack break
2. A stronger argument can be made to eliminate maturation and testing effects
c. Nonequivalent Control Group Design:
i. A quasi-experimental study that has at least one treatment group and one comparison group, but participants have not been randomly assigned to the two groups
ii. All members of group one experience the manipulation while all members of group two do not
iii. Nonequivalent Control Group Pretest/Posttest Design:
1. Version of this type of study as well that measures the subjects before and after there self selected group choice
a. Ex: self-confidence levels of women before and after
plastic surgery vs. a group of women who had no surgery
iv. Nonequivalent Control Group Interrupted Time-Series Design: 1. Independent variable was studied both as a repeated-measures variable and as an independent-groups variable.
a. Ex: Television access and crime rates
d. Types of Results:
1. The experimental group has a statistical difference compared to the control group between the pretest and posttest (ideally, there should not be a change in the control group)
1. Statistical difference between the pretest and posttest for the experimental group, but perhaps the control group started at a level before or below the experimental group.
a. Ex: The types of women who get plastic surgery might
have lower self-esteems in general while compared to
women who do not.
1. Control group and experimental group have data from pretest to posttest that are parallel.
2. Control group has scores that show no change pretest or
posttest but are still greater than the posttest or pretest for the
C. Types of Quasi Experiments:
a. Single Factor Design Without Manipulation:
i. One quasi-experimental IV (group membership)
ii. Levels of IV are preexisting
b. Single Factor Design with Manipulation:
i. All members of one group selected to be in one condition (ex:
ii. All members of another group selected to be in another condition (ex: control group)
iii. IV confounded with group membership
c. Mixed Factorial Design with Manipulation:
i. One between subjects quasi-experimental IV that is not manipulated ii. One within subjects experimental IV that is manipulated
d. Natural Experiments:
i. A form of Quasi-Experiments where the intervention (or assignment to conditions) happens due to external circumstances that are thought to be or close to random
ii. Used to evaluate causal hypotheses when true random assignment is impossible or unethical
iii. Make sure the groups are similar in all respects except for the factor that the researcher thinks is causal
1. Odds Ratio:
a. Measure of Association
b. >1 indicates an association/greater odds for the
Four Validities in Quasi-Experiments
i. Higher ecological validity, real world problems or context
ii. Can we generalize to other outcomes (DVs), conditions (IVs),
a. Selection Effects
1. Only relevant for independent-group designs, not for repeated
2. Applies when the groups at the various levels of the
independent variable leads to differences in the dependent
variables between the two groups
1. Within-subjects, or repeated measures design
2. Matched-Group Design (or analysis)
a. Using information on age, sex, BMI, income, and several
other variables to compare groups of people
3. Wait-List Design
a. All the participants plan to receive treatment, but are
assigned to do so at different times
1. Examine potential third variables to help rule out systematic effects
b. Maturation Effects
1. Occurs when, in an experimental or quasi-experimental design with a pretest and posttest, a treatment group shows an
improvement over time, but it is not clear whether the
improvement was caused by the treatment or whether the
group would have improved spontaneously.
1. Repeated Interrupted Time-Series Designs help rule out
spontaneous changes (two groups serve as a mutual control)
c. History Threat
1. Occurs when an external, historical event happens for
everyone in a study at the same time as the treatment variable. 2. It is unclear whether the outcome is caused by the treatment or by the common external event or factor
1. Control/Comparison Group
2. Beware of selection-history threats which will affect one group systemically
d. Regression to the Mean
1. Occurs when an extremely likely finding is caused by a
combination of random factors that are unlikely to happen again.
2. A threat primarily when a group is selected because of its
extremely high or low scores.
1. Longitudinal designs in which the researcher tests the participants repeatedly
e. Attrition Threat
1. Occurs when people drop out of a study overtime
2. Becomes an internal validity issue when people drop out of a study for systematic reasons.
1. Analyze dropouts vs. completers
2. Missing Value Analysis:
a. As long as the dropouts are not systematically different
from the completers, there is no attrition theat.
f. Testing/Instrumentation Effects
1. A kind of order effect in which participants tend to change as a
result of having been tested beforehand. Testing might cause
people to improve or get worse.
2. An instrument that changes how it measures over repeated use
1. Control/Comparison groups
g. Observer Effects:
i. Observer Bias: create a double-blind, or masked study
ii. Placebo: Include a control/comparison group
iii. Demand Characteristics: Were the participants able to detect the goals of the study?
a. Measure the effect size
i. (Cohen’s D)
ii. Looks at the how large the difference between the two groups were
b. Measure Statistical Significance
ii. Effect size and sample size required
D. Construct Validity
a. How well were the variables measured/manipulated?
A. Differences Between Small Samples and Large Samples
i. Participants are grouped
ii. Data from individual participants are not of interest in themselves; data from all participants in each group are studied together
iii. Data represented as group averages
i. Each participant is treated as a separate experiment
ii. Small-N designs are almost repeated-measure designs in which
researchers observe how the person or animal responds to several
systematically designed conditions
iii. Individual’s data is presented
iv. Instead of gathering a little information from a larger sample, they obtain a lot of information from just a few cases.
1. Single-N Designs restricts study to just one person/animal
B. Four Validities in Small-N Designs
i. Can be strong if properly designed
i. Problematic, depends on the “target” of the generalization
i. Strong when measures and observations are precise
i. Can be difficult, as for they’re to be statistical power, effect sizes need to be large.
C. Three Types of Small-N Designs
a. Stable-Baseline Design:
i. A study in which a researcher observes behavior for an extended baseline period before beginning a treatment or other intervention
ii. If the behavior is more stable during the baseline, the researcher is more certain of the treatments effectiveness.
iii. Longer pretest allows researchers to effectively say that treatment has a discernable effect and is not spurious
b. Multiple-Baseline Design
i. Researchers stagger their introduction of an intervention across a variety of contexts, times or situations
ii. Ex: Girl who repeatedly touches her hair, face, and objects is given a baseline for each. They target the first action, touching her hair, and
overcorrect the behavior for a few days
iii. Three behaviors act as control for each other to improve internal validity iv. Provide comparison conditions to which a treatment or intervention can be compared.
c. Reversal Design
i. Researcher observes problem behaviors both with and without treatment, but takes the treatment away for a while (the reversal period) to see
whether the problem behavior returns.
ii. If the treatment was really working, the behaviors should worsen again when the treatment is discontinued.
Ch. 14 – Replicability, Generalization, and the Real World
i. Can researchers get the same results, using the same data
1. Exact record of statistical procedure
2. Access to the same data
b. Is it Replicable?
i. Have the results been repeated, not do they have they potential to be
repeated. Gives the study credibility and it is crucial in the scientific
ii. Direct Replication:
1. Researchers repeat an original study as closely as they can to see if the original effects to see if the findings are replicable
a. Adequate descriptions of Experimental procedures
b. Same measures
c. Record of statistical procedures
d. New data
iii. Conceptual Replication
1. Researchers study the same research question but use different
2. The variables in the study are the same, but the procedures for
oprerationalizating the variables are different
a. New ways to define constructs
b. New data
1. Researchers replicate their original study but add variables to test
2. Might introduce a new situational variable
a. Ex: If degree of practice affected one’s ability to drive and
talk on the phone, IF a prior experiment had been
conducted to see how talking on the phone affected one’s
v. Replication by Independent Researchers
1. Many psychological scientists give extra weight to replication
studies that are conducted by independent researchers working
outside the lab
i. Often prospective studies (looks for disease onset and risk factors)
1. Researchers first raise a research question, forming a hypothesis
about the potential causes of a disease
2. The researchers then observe a group of people, the cohort, over a period of time (often several years), collecting data that may be
relevant to the disease.
a. This allows the researchers to detect any changes in health
in relation to the potential risk factors they have identified.
b. Used by epidemiologists looking into the factors that affect
the health and illness of populations.
iv. Led to the identification of risk factors for CVD3
C. Nested/Hierarchical Design
i. A class of experimental design in which every level of a given factor appears with only a single level of any other factor
ii. Real world interactions
b. Challenge to Analyze
i. Observations are not independent
ii. Special analysis techniques
1. Individuals within a classroom within public vs. private schools in
each of the fifty states
2. Family can be broken into income, housing condition, number of
family members etc.
3. Neighborhood can be broken down into safety, resources, grocery, park etc.
i. A way of averaging the results of all the studies that have tested the same variables to see what conclusions the whole body of work supports
1. Scientific Literature:
a. Consists of series of related studies by various researchers
ii. A set of statistical methods for quantitatively aggregating the results of several primary studies to arrive at an overall conclusion about either the relationship between two variables or the effectiveness of an intervention or treatment
1. Looks at overall effect size of multiple experiments
b. Myth of the Perfect Study:
i. There are no perfect studies
1. All studies contain measurement error
2. No study has measured the perfect construct validity
3. Sampling Error
ii. No single study or small selected subgroup of studies can provide an optimal basis for scientific conclusions
c. Scientific Need:
i. Studies often report conflicting information
ii. Individual study results interpreted based on statistical significance 1. Type II Error – (False negative) underpowered studies will not find statistical significance
2. Smaller sample sizes will result in smaller effects which will result in lower statistical validity
3. Strength of Relationship:
a. Cohen’s D
i. Weak = .20
ii. Medium = .50
iii. Strong = .80
i. Weak = .10
ii. Medium = .30
iii. Strong = .50
d. Strengths and Limitations
1. Peer reviewed more scrupulously as each article it contains was
first published in an academic journal
2. Aggregate large volumes of literature
3. Control for sampling error and measurement error
4. Focuses on magnitude of the effects
5. Resolves conflicts in literature
6. Test moderators
7. More standardized and objective method
8. Generate population parameters
9. When one combines multiple studies, it increase the sample size
and allows researchers to identify small effects/trends with
1. File Drawer Problem:
a. There is a publication bias as significant relationships are
more likely to be published than null effects
b. Meta-Analyses might be overestimating the true size of an
effect because null effects or even opposite effects have not
been included in the collection processes
c. Can address partially by open publication of null
results/failure to replicate the studies
d. Can request file-drawer data from colleagues
2. Too many judgment calls/subjective decisions
3. Might be comparing apples and oranges
a. Can address this by including differences in moderators
4. Garbage In-Garbage Out
a. Can address by weighting studies based on quality and
iii. Good Judgment is Essential
1. Judgment is needed at all steps of the research process:
i. Think about expected effect sizes and design studies
with enough power that they can reasonably find
ii. Minimize measurement error and maximize sample
i. Interpretation should be based on strength (validity,
reliability, scientific rigor)
E. Combining Studies
a. Assigning Value
i. Not all studies have equal value
1. Some have stronger methods, some have larger samples
2. Quantify the quality of each study to determine the weight given to each study
ii. Weight studies based on quality
iii. Compare multiple studies based on effect size
iv. Code other relevant moderators or design factors
b. Conducting Meta-Analyses
i. Identify Research Area/Question
1. Ex: “The ordinal effects of ostracism”
ii. Figure Out Inclusion/Exclusion Criteria
1. Ex: “We only considered cyberball experiments”
iii. Locate Studies
1. Ex: “We used seven search strategies from 2012 to 2013”
iv. For each study, compute focal effect size. Record any other variables (potential moderators)
1. Ex: “We used hedges “g” version”
v. Compile Database
vi. Complete Analysis
F. External Validity
a. Generalizing to other Participants
i. Must have a probabilistic sample of the population
ii. Understand that the population of interest isn’t generally the entire population
iii. “How” matters more than “how many”
b. Generalizing to other Settings
i. Ecological Validity:
1. “Mundane Realism” a study’s similarity to real world contexts
ii. The importance of external validity depends on a researchers priorities iii. Cultural Psychology:
1. Sub-discipline of psychology that focuses on how cultural contexts shape the way a person thinks, feels, and behaves
2. Muller-Leyer Illusion
c. Real World
i. Field Setting
1. When a study takes place in the real world
ii. Experimental Realism
1. Many laboratory experiments create settings in which people
experience authentic emotions, motivations, and behaviors
G. Scientific Mode:
a. Theory Testing Mode
i. Researchers usually are testing the association or causal claims to
investigate support for a theory
ii. Theory-Data Cycle is the process of designing studies to test a theory and using the data from studies to reject, refine, or support the theory. In this mode, external validity matters less than internal
b. Generalization Mode
i. Researchers want to generalize the findings from the sample in their study to a larger population