310 Test One Study Guide
310 Test One Study Guide 3100
Popular in Advanced Experimental Psychology
verified elite notetaker
Popular in Psychlogy
This 8 page Study Guide was uploaded by Grace Gibson on Wednesday February 3, 2016. The Study Guide belongs to 3100 at Clemson University taught by Dr. Thomas Britt in Winter 2016. Since its upload, it has received 255 views. For similar materials see Advanced Experimental Psychology in Psychlogy at Clemson University.
Reviews for 310 Test One Study Guide
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 02/03/16
Overview of Research Methods and Process How do we know something? ● believe what authority is telling us ● deduction: logical proof (e.g. women are caring and Susan is a woman so Susan is caring) ● induction: observe nature and coming up with something to test (often underappreciated because we don’t always observe; we assume instead) ○ induction is credited to Francis Bacon ○ Four Steps: ■ observe nature ■ develop theory and hypotheses based on observations ■ collect data to test your theory ■ come up with generalizations after multiple studies ● hindsight bias: we assume we could know the results of the study before it’s carried out ● terror management theory: we all know the world is going to end and we do things to curb that anxiety Four Hallmarks of Science ● phenomena we observe are lawful (result of identifiable causes) ● a theory is developed to account for a given phenomenon ○ a good theory accounts for findings we already know ○ a good theory is parsimonious and leads to additional research ● hypotheses from theories are open to disconfirmation ● results are published for scrutiny by peers The Scientific Approach ● theorized construct: a hypothesized variable used to explain a domain of thought, behavior, or emotion ● operationalization of a construct: steps or operations that define a construct ○ correlation is measuring a construct ● you can measure a construct or manipulate a construct ● correlational research: measure two or more variables and examine the relationship ○ variables may be continuous or categorical ○ assesses strength and significance of the relationship between the variables ○ multiple correlation: correlation between one variable and a set of other variables ○ don’t dismiss correlation just because you can’t infer causality because it isn’t always ethical to manipulate variables ○ sometimes correlational research is the only ethical option ● experimental research: the key conditions are manipulation and random assignment ○ independent and dependent variables ○ assess manipulation intervention ○ allows causal inferences to a point ○ can you have the same participants in different conditions or do you need different people? ○ are experiments representative of society? ○ how likely will the results generalize outside the lab? ■ e.g. does white noise in the lab generalize to conversational noise in the office? ○ within subject: same participants in all conditions ● quasiexperimental design: you take advantage of a naturally occurring independent variable that you don’t manipulate ○ effects of 9/11 on approval of law enforcement, Magic Johnson coming out and public opinion on AIDS, etc… ● single case analysis: Freud, behaviorists, humanists, and physiologists ○ good for getting ideas for more research ● metaanalysis: you look at many studies done on one subject to make your hypothesis ○ combining effect sizes across studies to test hypotheses ○ Hyde found that over a hundred studies and did a metaanalysis of gender and aggression ○ she found that men were generally more aggressive than women, especially when it came to behavioral manifestations of aggression ● qualitative research designs: in depth interviews and focus groups for rich information How to Generate a Hypothesis ● read journal articles and see how you can contribute ● get ideas from poetry or novels ● look at established findings and think about where they will not fold ● think about puzzling behavior and why a person does it (selfhandicapping masochism) Variability ● variability: what we’re all about ● if there’s no variability, there’s not study because there’s nothing to explain ● you could look at how variability of one construct accounts for variability in another construct ● variability is the difference between and within people on a theoretical construct ● psychology is always trying to explain or measure variability in something ● in an experiment, you’re trying to find if your independent variable accounts for a portion of that variability ● progress in the field of psychology is gaged by the amount of variance explained in important dependent variables ● there are differences between people such as GPA, selfesteem, job performance, etc… ● there are differences within people such as selfesteem, energy, extraversion, etc… ● correlational research: how much variability in one variable can be accounted for in another variable? ○ r^2 = variability of one variable as accounted for by another variable ○ only 16% of the variability in GPA can be determined by SAT scores ○ R^2 = percentage of variability in one variable accounted for by two or more variables ● experimental research: how much variability in the dependent variable is due to the independent variable? ○ eta^2 = how much of total variability in the study is due to the independent variable? ● metaanalysis: interested in explaining variability across studies ○ what predicts whether studies will produce a large or small effect ○ lab vs. field studies? The HypotheticoDeductive Process ● derive hypotheses from a plausible theory ● collect data to test hypotheses ● indicate whether data supports or refutes hypotheses ● discuss implications for the original theory and beyond HARKing ● this research process is not always as straightforward as the HD process ● sometimes the research is more exploratory (no set hypotheses) ● results may not support your hypothesis but you realize they make sense according to another theory ● question is, do you write a paper on this? ● hard core purist: indicate in the discussion that the results turned out different than you anticipated ● hard core revisionist: rewrite your introduction given what you know about your results ● HARKing: hypothesizing after results are known (presenting a posthoc hypothesis in the introduction as if it were a priori hypothesis ● empirical inspiration: type of harking where the results reveal a hypothesis you should have thought of before ● omit loser hypotheses: type of harking where you don’t include the hypotheses that weren’t supported (if you started with multiple hypotheses) ● simple harking: you let the results completely guide your introduction ● Kerr sent a survey to other psychologists and asked them how much harking they did and how much they thought their coworkers did ○ the classic HD approach and empirical inspiration were what happened most often ○ simple and omit loser harking occurred as well, but less frequently ○ psychologists thought their coworkers did empirical inspiration and simple harking more often than the classic HD approach ○ psychologists thought researchers should morally do empirical inspiration or HD ● Incentives for harking ○ psychologists are rewarded for publishing so they want to tell the best story they can ○ confirming a priori hypothesis is seen as superior to disconfirming a hypothesis ○ if a paper does not have a strong theoretical backing, it is seen as weak ○ journal space is too precious to waste talking about your “wrongheaded hunches” ○ there is emphasis on telling a good story for an article to be published ○ hindsight bias: results seem obvious once you see the data; you should have predicted the effect ● Reducing harking ○ judge the experiment just on the introduction, hypotheses, and method, not the results ○ change the attitude about harking ○ educate students about the costs and benefits of harking (most undergraduate students don’t learn about harking) ○ promote replicating results that way the second time around you can start with the post hoc hypothesis ○ make harking a basis for rejecting an article; this could be done by requiring people to turn in their hypotheses before the experiment ■ signs of harking to watch out for are if there isn’t really a theory guiding the paper or if everything comes out significant or really complicated ○ legitimize exploratory research by admitting that many times you cannot make a strong hypotheses in a specific direction ○ allow researchers to admit up front that the hypotheses were post hoc ○ requires change in the whole publication approach MEASUREMENT METHODS Subjective Data ● describing an experience with a number ● you can ask standard questions so replication is easy ● agree/disagree, checklists, rankings, frequencies, etc… ● this is usually used to find personal qualities like personality, skills and abilities, life satisfaction, political attitudes or behaviors, substance abuse, job performance,etc…. ● in the NASA journals program, there were unlikely/likely questions and open ended ones ● you can also find things out about other people (e.g. teacher evaluations) ● in clinical psychology, a clinician might complete their own ratings of the client’s mental state ● you can look at other’s personality traits ● Strengths: ○ standard overall measurements that can be used over and over again ○ operationalizes constructs that are hard to measure objectively ○ it’s easy (fast, flexible, easy to construct, inexpensive) ● Weaknesses ○ people might be worried about what people think of their answers ○ people may not want to admit the truth to themselves ○ questions may not test what you want it to ○ you may not know yourself well enough to properly answer questions ○ answers can be influenced by mood or personality ○ instructions can be unclear or ambiguous questions Observational Ratings of Behavior ● exactly what it sounds like; you observe behaviors ● you can do this through experiment or correlation ● you can count up instances of particular behaviors ● Strengths ○ no self report bias ○ does not require responses from participants ○ more valid because it’s in a realistic environment ○ can be more objective ○ good for behavioral measures ● Weaknesses ○ people will behave differently if they know they are being observed ○ hard to do over a long period of time ○ time and labor intensive ○ behavior can be linked to a particular setting ○ not useful for assessing thoughts or feelings Physiological Data ● psychologists can often measure conflicts of interest by looking at physiological things ● heart rate, skin conductance, brain waves, cortisol in saliva, etc… ● actigraphs: strap on the wrist and it measures sleep and daily movement (frequently used to objectively measure sleep) ● blood pressure and heart rate are used to measure stress ● cortisol and testosterone in saliva are used to measure aggression ● fMRIs measure blood flow in the brain and tells us which brain area is responsible for different activities ● Strengths ○ very objective ○ very observable ○ direct measurement ○ less likely to be biased ● Weaknesses ○ expensive ○ contextual effects: you can have different physiological symptoms in different places ■ white coat hypertension: high blood pressure only at the clinic ■ occult hypertension: high blood pressure outside the clinic, but it goes down at the doctor ○ time investment to learn to use equipment ○ expertise is required Indirect Measures ● Project Implicit tried to measure reaction times to positive and negative paired to black and white faces ● is it faster to identify a word that’s negative when it is paired with a black face? ● the idea is that if you’re showing an implicit racism, it will take you longer to associate a positive word with a black face ● there can be a lot of error variance that has nothing to do with what you’re measuring ● this kind of test can be used to measure selfesteem or attitudes to a different target ● another example: people were asked to rate letters of the alphabet in terms of attractiveness ○ the idea is that people with higher selfesteem will think the letters in their name are more attractive ● can be good because there are some things you just can’t measure directly Archival Data ● data that already exists ● usually gathered for reasons other than what you want to do ● used for secondary data analysis ● public records, research organizations, company records, media reports, social media activity, etc… ● Strengths ○ usually there are large, complex sets of data available ○ don’t have to worry about ethical concerns ○ often less expensive ● Weaknesses ○ information could be outdated ○ records may not be accurate ○ it can be hard to fit these measures to theory ○ data may not be collected in the best way it could Importance of Converging Operations ● it is better to have converging operations, or multiple measures of one construct ● measurement pluralism: use different ways to measure a construct (different operationalizations that get at the same construct) ● in WWII we needed a way to choose leaders in the military ○ everyone that came through gave selfreports of leadership ○ they got peer ratings of leadership ○ created leadership scenarios and observed at how people led the team ● this is not often done because it is a huge pain ● Multitrait/Multimethod Matrix (MTMM) ○ look at correlations for different methods assessing the same construct ■ convergent ■ you can these to correlate highly ○ look at correlations among different constructs with the same method of measurement ■ divergent ■ you want these to correlate low because you don’t want the method to be the thing influencing these correlations Assumptions of the Classical Test Theory ● O = T + E (observed score = true score + error) ● true score: where the person would stand on the construct if it were measured without error ○ this is impossible to assess; you can only minimize error) ● error variability: factors other than the true score that influence the individual’s observed score ○ includes systematic sources of error such as intelligence or response sets ○ includes random sources of error such as mood, recent experiences, etc…. ○ error variability harms the predictive validity of the test Assessing Reliability ● reliability of a test: internal consistency ● Cronbach Alpha: statistic that tells you how well your measures correlate with each other ○ do items in your survey lead to results that correlate well ○ literally average correlation of each item with total score ○ typically you want it to be greater than 0.7 ○ a larger number of items leads to a larger alpha ● Test/Retest Reliability: if you administer the same test to the same people twice, those scores should correlate highly ○ correlation depends on the amount of time between tests ○ correlation depends on the nature of the construct being measured (intelligence vs. mood) ○ you want this correlation to be greater than 0.7 ● Alternate Forms Reliability: if you have two forms that test the same construct, there should be a high correlation ○ you want this correlation to be greater than 0.8 Assessing Validity ● validity of a test: does the scale measure what it is supposed to measure? ● content validity: identify different areas of the construct to be examined and make sure that the questions tap all areas of the construct ○ this is easy with academic tests ○ you have to find a balance between being too narrow and being too broad ○ e.g. GPA and graduation status is not enough to determine whether a student has performed well or not ● face validity: on the face of it, do the items appear to be getting at the construct? ○ you’d be surprised at how often this isn’t true ○ there’s no hard or fast numbers to measure this ● convergent validity: does the measure correlate with theoretically related variables? ○ e.g. do mental health scales correlate with resilience scales? ● divergent validity: your measures should not correlate with variables it’s not related to ○ the main thing you typically want to show is that your measure does not correlate with social desirability and intelligence ○ everyone wants to be resilient, so you want to prove that social desirability aren’t causing your answers ● known groups validity: do groups that should differ on the construct, differ? ○ e.g people in greek life should be more extraverted than the math club ○ e.g. people suffering from illness with a negative attitude should score lower on resilience than people suffering from illness with a positive attitude ● criterion related validity: does your scale predict theoretically related behaviors? ○ observable behaviors that are not based on self report ○ conscientiousness vs. course grades, sale numbers, GPA, etc… ● predictive validity: does the measure predict future outcomes, whether these are behavioral or assessed through different means? Nomological Network ● a nomological network is how variables within a research field are related to one another ● brings together all the forms of validity ● e.g. there are many studies looking at personality and health ○ there are many different variables between personality and health ○ you need to put your new variable in this nomological network and see how it relates to other variables already in there ● you need to show that your variable is not just a function of other variables in the nomological network
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'