New User Special Price Expires in

Let's log you in.

Sign in with Facebook


Don't have a StudySoup account? Create one here!


Create a StudySoup account

Be part of our community, it's free to join!

Sign up with Facebook


Create your account
By creating an account you agree to StudySoup's terms and conditions and privacy policy

Already have a StudySoup account? Login here

Research Design & Analysis: Lecture 15

by: Kimberly Notetaker

Research Design & Analysis: Lecture 15 PSY 3392

Marketplace > University of Texas at Dallas > Psychlogy > PSY 3392 > Research Design Analysis Lecture 15
Kimberly Notetaker

Preview These Notes for FREE

Get a free preview of these Notes, just enter your email below.

Unlock Preview
Unlock Preview

Preview these materials now for free

Why put in your email? Get access to more of this material and other relevant free materials for your school

View Preview

About this Document

These notes cover the material discussed in lecture (Quasi-Experimental Designs) for the week of 4/18.
Research Design and Analysis
Noah Sasson
Class Notes
research, Design, and, analysis
25 ?




Popular in Research Design and Analysis

Popular in Psychlogy

This 8 page Class Notes was uploaded by Kimberly Notetaker on Monday April 25, 2016. The Class Notes belongs to PSY 3392 at University of Texas at Dallas taught by Noah Sasson in Summer 2015. Since its upload, it has received 14 views. For similar materials see Research Design and Analysis in Psychlogy at University of Texas at Dallas.


Reviews for Research Design & Analysis: Lecture 15


Report this Material


What is Karma?


Karma is the currency of StudySoup.

You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 04/25/16
QUASI-EXPERIMENTAL DESIGNS “Quasi”: To some degree, in some manner, virtual, in some significant sense or degree; an “almost” experience - Quasi-experiments are used most often in applied (i.e., natural) settings The True Experiment  A manipulation (only one level of the IV gets it)  An appropriate comparison group (the other level of the IV)  High degree of control o Randomization o Control of all conditions of the experiment o Lack of full control distinguishes the quasi-experiment from the true experiment The Quasi-experiment  Has an IV and a DV  Includes a comparison group, but lacks full control o No Randomization – uses “nonequivalent control group”  If at all possible, use a true experiment Problems with Conducting a True Experiment in Real World Settings - Obtaining permission from people in authority - Obtaining participants o Typically, not all individuals want to participate  Self-selection bias - Randomization o Often difficult in applied settings  Can be disruptive  Perception that randomization is “unfair”  Ethics of the no-treatment control If a no-treatment control group is unfair, what are the alternatives? - Alternate order of treatments o One group gets tx1 then tx2, other gets tx2 then tx1 - Wait-list control o One group gets tx; if beneficial, control group gets tx later - Use of established treatments as a control - Treatment as usual control (the least you can do; free to pursue whatever treatment you’d like) Why Use a Quasi-Experiment?  Randomization not always feasible  External validity – does the intervention “work” in the real world  Some hypotheses can’t be practically tested with a true experiment o Examples: 1. Testing differences between natural groups 2. Testing the efficacy of public program (e.g., a new recycling initiative) or social policy (e.g., abstinence-only education) Threats to Internal Validity  Confounds that provide alternative explanations for a research finding o Controlled by the true experiment…but not always a quasi- experiment  The quasi-experiment is a compromise  Eight main “threats to internal validity” o a.k.a., confounds or alternative explanations o All of these threats are present in the pre-test/post-test designs that I detest so vehemently  The Eight Common Confounding Effects: 1. History: some non-tx event produces change  Ex: treating alcoholism; solution: control group 2. Maturation: people change over time  Ex: treating depression; solution: control group 3. Testing: people get better when tested again  Ex: measuring IQ in an intervention for a learning disability; solution: space them far apart; use equivalent but different measures; control group (what a placebo in a drug trial does) 4. Instrumentation: measure of your DV changes  Ex: mechanical changes (scale), or interviewers get better; solution: ensure reliability and validity of instruments, control group 5. Regression to the Mean  Any individual score may be invalidly high or low due to chance and not by an accurate measure of performance  People at extreme ends of a measure tend to move toward the middle over time  Ex: improving memory; baseball batting averages o Regression to the mean can easily be mistaken for a treatment effect!  Solution: don’t choose samples based on the “extreme” scores on a pre-test; use reliable measures 6. Subject Attrition: participants are lost over time  Ex: people lose interest in study, move away, etc.; final sample then differs from the beginning sample  Solution: careful follow-up procedures; statistically compare those who dropped out to those who remain (look at the pre-test to those who dropped out…) 7. Selection: One group systematically differs from the other in ways unrelated to the intervention  Ex: one floor of a mental health hospital gets the tx, the other is the control group.  Solution: randomization; if not possible, remain aware of this threat and try to match groups to minimize confounds 8. Addictive Effects with Selection: when any of the first 6 threats exists for 1 group but not the other (#1 problem in a quasi-experiment) Limitations of the True Experiment:  While true experiments control for these 8 threats, they still can’t control everything o There are a few universal threats to internal validity that can affect even true experiments!  Contamination – when groups communicate o Diffusion of treatments – tx transfers to conrols  All controlled if groups are kept SEPARATED (not easy) o Resentment and Rivalry  Experimenter Expectancy o Controlled through double blind studies More Limitations…  Novelty Effects – the “newness” of the tx has an effect rather than the tx itself o Example of pink prison cell study Evaluating Public Programs and Policy  One-group pretest-posttest o You know how much I hate this one!  No control group at all o Just two observations, with the treatment in the middle o Design diagram: O 1 X O2  A very, very bad choice! o No control group = no definitive conclusions! o All threats to internal validity may be present! Nonequivalent Control Group Design  Compare intervention to a “like” control group, but without randomization » “Like”-ness determined by pretest match on the DV  If groups truly are comparable, it controls for many threats to internal validity (but not additive effects with selection)  Still Vulnerable to “Additive Effects with Selection”! Interrupted Time Series Design  Comparison of baseline before and after intervention  Better than pretest-posttest » because it uses many observations before (baseline) and after the intervention » Good for one time interventions where long-term effects are desired  BUT no control group! So not an ideal design. Time Series with Nonequivalent Control Group - A better option - Multiple pre and post tests - Intervention plus Comparison group - Rules out history and instrumentation effects COMMUNICATING RESEARCH What is different about this course than all the other ones? It’s an application course! Conducting Research - Idea or Question o Hypothesis - Devise way to investigate this idea *this class covers o A method to test your hypothesis - Examine and interpret what you find o Have your results answered your questions? - Tell other people! o The most important! The Research Process (8 Steps) Disseminating Research  Presenting at research conferences o Lectures o Poster presentations  Colloquia and Lectures o Center for Children and Families Lecture series o DCS: Developmental, Cognitive, and Social/Personality brownbag series  Research reports o Peer reviewed – provides validation Writing Research Reports  Purpose of the Research Report: o Standardized format for a scientist to explain to readers about their research o Why you did this?  Make a case for your study o What did you do?  Describe your methods o What did you find?  Results of the study o Does any of this matter?  Explain the findings and what they mean  APA Style o Purpose:  Common organizational format  Guidelines for how to present research in a clear and unbiased manner  Readers and writers alike benefit from a standardized format  Structure of the Report: o Abstract  Summarize  Issue  Method  Findings  Conclusions and Implications  100 – 150 words (…or longer depending on journal rules)  Note: Although this is the first section read, it is often written last o Introduction (answers the question: why you did this study)  Brief – comprehensive but not exhaustive  Do not bore your readers with excess information  Why is your study different/important  Describe the problem and why it is important  Make a specific case for your study, not just a summary of previous research o Selective presentation to support your specific argument based on past literature with citations  Present hypotheses  Must be justified by the literature o Method  Participants  How many? o Sample size: N = ?  Who/what?  What were the characteristics of the participants? o Human Age, ethnicity, SES, other relevant variables o Animal: Species, source, etc. o Archival: Source  Measures/Materials  What kind of data was collected? o Questionnaires/Survey Data  What do they measure? Are there sibscales?  Types of scales and reporting  Some example items o Observational data  Describe where and how  Coding systems?  What is being coded? Who is doing the coding?  Operational definitions of all relevant variables  Independent/Dependent  Procedure o How were the data collected?  Data collection protocol; step by step procedure  Recruitment o SONA, flyers, craigslist, etc.  IRB approval and informed consent  Participant tasks  Data Analysis (sometimes) o Only if analytic techniques were particularly complex o Explain and justify the reasons for using such complex analyses  Goals of Method Section:  Reader knows exactly what happened  Detailed enough that other researchers could replicate o Results – what happened  Present and summarize analyses – don’t interpret them  Include:  Purpose of the analysis  Type of analysis  Summary of analytic results  Present confidence intervals, effect sizes, etc.  Summary statement  Results for each hypothesis plus post hoc tests  Tables and Figures  Use sparingly o Only when information cannot be conveyed easily or in another way  Do not repeat information  Refer the reader to the important results in a table/figure  Be careful of misleading scale in figures o Discussion – what it means  Summarize results briefly  Which hypotheses were or were not supported?  Were the results as expected or were there surprises?  Discuss the meaning/interpretation of results  What is the meaning of these specific results?  How do these results generalize beyond the sample (if they do)? *samples generalize  What do the results mean for science and theory more broadly?  Be conservative; don’t go beyond the data  Causal statements  Excessive theorizing  Limitations:  All studies have them!  Note limitations  Explain how the results can be generalized and how they can’t  Discuss how future research might correct current limitations  Purpose of discussing limitations?  What next? Conclusions and Implications  Brief (2 – 3 paragraphs at most)  Broad importance of the study  Application and implications for future research o References  List everything you cited!  Be careful of citing presentations, dissertations, and non peer- reviewed material o Why? o Your work may be seen as weaker  Flaws in theory or ability to find solid support for your hypotheses Full Circle  Explained purpose and rationale of study based on previous literature, and outlined hypotheses to be tested o Intro  Detailed how the study was conducted in order to answer the questions posed in the intro o Methods  Describes what was found using the data gathered through procedures outlined in the Methods o Results  Interprets the Results and relates them to current knowledge and the real- world o Discussion


Buy Material

Are you sure you want to buy this material for

25 Karma

Buy Material

BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.


You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

Why people love StudySoup

Steve Martinelli UC Los Angeles

"There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

Amaris Trozzo George Washington University

"I made $350 in just two days after posting my first study guide."

Jim McGreen Ohio University

"Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."


"Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

Become an Elite Notetaker and start selling your notes online!

Refund Policy


All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email


StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here:

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.