### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Research Methods PSYC 512

UI

GPA 3.5

### View Full Document

## 51

## 0

## Popular in Course

## Popular in Psychlogy

This 57 page Class Notes was uploaded by Dorris Purdy on Friday October 23, 2015. The Class Notes belongs to PSYC 512 at University of Idaho taught by Brian Dyre in Fall. Since its upload, it has received 51 views. For similar materials see /class/227923/psyc-512-university-of-idaho in Psychlogy at University of Idaho.

## Popular in Psychlogy

## Reviews for Research Methods

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/23/15

PSYC512 Research Methods Brian P Dyre University ofIdaho stcaiz Research Mam Lecture 18 Outline Research Proposal Information Writing Research Proposal The Art of Peer Review Addressing Reviewer39s Comments Inferential Statistics Testing for differences Determin39ng W a sample repraents a population Demrmin39ng iftwo or more samples differ esting for relationships of relationd1ip Shength of relationd1ip stcaiz Research Mam Writing a Research Proposal Purpose of Research Propos Present a literature review defining concepls and intervening variables pertinent to research question Present a specific research question with explicit hypotheses to be tested Develo a plan for addressing that research question empirically including descriptions of Target population and subject sampling The research desi n including explicit definition of 39 en nt varia les stimuli and procedures implement these variables Aspects of stimuli and procedures that control for extraneous and confounding varia les Types of analyses to be used predicted resulls and how these predictions relate to the hypotheses stcaiz Research Mam Research Proposals Global Concerns Scientific Writing Style 39 39 more important than entertainment but Research is part science and art advertising not only do you need to develop good ideas but you must be able to sell your ideas ften the basis of first impression for the quality of the research roject Proposal must be clear on ALL levels of analysis ords Sentences Paragraphs OUTLINE Sections 2 a E n39 stcaiz Research Mam Research Proposals Global Concerns APA guidelines Orderly expression of ideas organization Smoothness of expression transitions Economy of expression concise language Precision and clarity use scientific vocabulary jargon correctly and insure that all terms are defined the first time they are used stcaiz Research Mam Organization of a Research Proposal APA STYLE Title Page title should specifically describe what the paper is about so that it is useful information for other researchers39 literature searches stract essentially a minipaper for lit searches extremer CONCISE lt 150 words introduce specific topic discuss variables etc present major results no statistics discuss important conclusions stcaiz Research Mam Organization of a Research Proposal APA STYLE Introduction Pur se Demonstrate knowledge of relevant research Define intervenin variables and their relation to manipulations an measuremenis used In preVIous Present and justify research question and hypotheses Present and justify the general method to be used Organization Start broad then narrow to your general purpose Discuss only relevant research in a logical flow Organization of a Research Proposal APA STYLE Method explicitly state how variables are manipulated define in separate subsections Participants subjecls Design StimuliApparatusMaterials rooedures esulls Describe the Scales of IVs and DVs N th d d t t he t Describe Transformations of DVs ear een provi ean expici s a men 0 hypotheses and an overview of the general research 39 summar39ze planne Stat39St39cal analyses sign Summarize predictions stcaiz Newman stcaiz Newman Important Elements of Style Important Elements of Style Brian s pet peeves Define scientific terms jargon and abbreviations at first use then use terms consistently Do not use informal or colloquial language eg you don39t run subjecls you test subjecls Ambiguous pronouns if you use the word it ake sure the surrounding context makes the meaning of the word it obvious otherwise avoid using it as in itquot Sexist pronouns he vs she having to use these if at all word sentences to avoid possible or use he or shequot stc iz Wigwam Brian s pet peeves Verb tense Introduction Section Discussion of specific previous research 9 past se Discussion of ideas or concepts that are general or all time and not linked to specific moment in research report past tense stc iz Wigwam Important Elements of Style Brian s pet peeves Pluralsingular mismatches assive voice word sentences in active voice Superfluous imprecise language eg avoid vague adverbs search or all words ending in ly quotand consider eliminating em Style issues like thatquotvs which For more information on writing styleI recommend StmnkWjrand White EB 2000 The Elements ofStye 4m Ed Boston Allyn and Bacon stc iz Wigwam The Art of Peer Review Example review available online in lecture schedule Goal Assist author in improving the clarity and impact of the paper or proposal by offering specific constructivecriticism no name calling and complements where appropriate Stmcture Summary Major or general criticisms typically lt3 Minor specific criticisms any number specifically listed by page and line number stc iz Wigwam Addressing Reviewer s Comments Never blame a reviewer for a negative rev39 Consider all criticism as constructivehthe reviewer is trying to help Assume misunderstandings are your fault not the reviewers All misunderstandings occur because you did not write clearly enou h The cover letter required with submission of revised proposal Speci fically responds to the reviewers comments by describing specific changes made in the paper to address critic39 Presenting a rationale for why a reviewer39s criticism was not addressed Must be worded very diplomatically stc 5 Resulch Mum Presenting Research Timeallotted 20 minutes 9 strictly enforced Us same general format as the written report Given time a c you cannot go into the same lev lof detail as yourwritten report Ma terials Talk from an outline of points you wish to make Visual Aids 9 powe 39 Elements of Style PRACTICEYOUR PRESENTATION Anticipate questions a stc nd how you will answer them as 2 Resulch Mu Using Inferential Statistics I Which Statistic The statistical decision tree Howell Figure 11 Testing for relationships vs differenoes a false distinction Relatio 39 39 rength of relationship bet bles Differences comparing different groups or treatmenis on some measurement But what causes those differences The relationship between the independent variable defining the gro rea ment a e ependent variable Hence testing for differences is really testing the relationship between the IV and DV stc lz nestmutlmg Analyzing Frequencies Howell Chapter 5 Bernoulli Trials series of hdwmdmt triab that rallt in one of two mutually exclusive outcomes Eg co39n i s gende of babies bom 39ncrease of decrease in a mealte a application of a treatment The B39nomial Dishbution stc lz nestmutlmg Analyzing Frequencies Howell Chapter 5 Usrlg me blrlomlal dlsmbuu n Mean rlurrber of successes All Valance ll l rlurrber of success NW Tesurlg ypotlnesesusng the blnomlal dlsmbuuorl The slgn Test Ho lstyplcallyp q that do 39 0 50750 cnance of excess of falure but emthave to be unecase H1lstyplczllyp7q Plu in values for N x p and q and DPO alrech provldes me Ere ablllty mat me patten of data oou d result glvel39l the null ypodnesls ls true Sum me probablllues 300 for all rlurrber gt x to get me total probanlltyomndlng pgtX important The Slgl l test takes ll39lto aocourlt dlrecu on of dlffererloes but not magnlmde stc lz nestmutlmg What about multiple more than 2 possible outcomes Analyzing Frequencies Howell Chapter 5 Multinomial distribution stc lz nestmutlmg Analyzing Frequencies Howell Chapter 5 Usi ig the mulunomial dismbuuoi i Mean Xk t Vanance ll i Xk Alek1pXk Tesurig Hypotheses using the rruluriomal distribution Ho istypicall bu wk 11ltach outcome has the same chance but at doesn39t have to be the case Hlistypiczllyp 7w 7 k Plug ll i values for Ngtlta1d 13 and bgtlt1gtltZ gt9 dnecdy provides the probability that this parumlar pattern of data oould result given the null hypothesis is tme lVList Sum the probabilities tot aii battens that deviate equal to or more m get the total probdoility e urne consumngi stc iz ResuichMethads Analyzing Frequencies Howell Chapter 6 EasierAltemative to Multinomial distribution Chisquare xztest Compare computed value k is the runbe of of x2 to value of x1 categories in the varid e distribution with dfk1 a is the observed frequency Expected frequencies for for eaCh CBEQOVY the n H hypothesis Eis the expecmd frequency typically Nk where N for each camgory is the total number of iis the category 39ndex observations stc iz ResuichMethads Analyzing Frequencies Howell Chapter 6 Using yzwiih trult39ple dimensions contingency 3525116233332211 R is the numbe of camgoties 39n 539 the dimension def ned by the the othe dimension rows of the mbie c is the numbe of camgoties in 7 RC the dimension def ned by the N is the total mmbe of columns of the mbie observations 0 is me obseved frequency tot o conpumd tape of y2 ead1 camgot m vane of xzdis bu on Wi39h Eis the ex ecmd tre uetc tot dfR101 eadt eEgoty q Y i a td i ae category 39ndices stc iz ResuichMethads Analyzing Frequencies Howell Chapter 6 Assumptions of the x2 test Each observation is independent Inclusion of nonoccurrences stc iz ResuichMethads PSYC512 Research Methods Brian P Dyre University ofIdaho stcaiz Research Mum Lecture 3 Outline Overview of the Research Process Searching the literature Reading the literature Epistemology and Scientific Explanation stcaiz Research Mum The Research Process Develop a research idea and hypothesis Develop idea into a testable h thesis 3 Choose a research design Experimental Correlational Quasirexperimental 4 Obtain subjects sampling ethical and practical considerations The stdd steps form the bass of your research pm dsd for W class ad a mess prOpOSa should you decide m amp ete one Conduct the study Analyze the results Report your resulls Conferences Journals Repeat Steps 17 forever stcaiz Research Mum Searching the Literature Electronic Resources he UI Library electronic databases Google Scholar beta Social Science Citation Index SSCI available online at WSU Library Hardcopy Resources the old traditional wayquot Textbooks 9 identify seminal research papers use SCI to search fonvard in time for more recent works that were influenced Reference sections of papers 9 trace back in time to revious research that influenced the current work stcaiz Research Mum Searching the Literature General Strategy Initially obtain a broad overview of a particular area reading review papers or books and seminal works rather than tightly focused empirical reports 9 helps to organize understanding and identify key issues Once broad overview of a research area has been obtained delve into more tightly focused empirical re rovides more nuts and boltsquot methodological information and data stcaiz Research Mum Organizing Reading and Most Importantly Remembering the Literature Never read passively Always have a goal in mind obtaining some particular piece or type of information Observe your circadian rhythms for periods of s and take advanta Never read without taking notes to include in your annotated bibliogra hy Plumb the reference section for further readings stcaiz Research Mum Writing an Annotated Bibliography As you read cream a1 outiine of the general area of 39nqu39r tlat is organized by irgoortant isles emp39rical rants theoretica stances hypotheses a1 questions a heps m organize understmd39ng and identify key issies In your outi39ne 39ncIJde entries for every paper you have found that oontahs the full citation perth inctiding the abstract for the most relevmt pages and a petrl39nk m the full Ext if available electronically mine or on a local dHlt Sometimes you may WH1 m include references m a particula paper in nultiple plaoes in your outihe Ead1 citation should be followed by detailed noms of key isles ra39sed by each paper eg hypothaes and rationale general method most important results and your own comments relation of work m other raead1 and your own remrch project relative inportanoe of work followup questions Psvc iz summaries Using Your Annotated Bibliography Think of your annotated bibliography as your memoryquot of the research in your field of interest update and refer to i 0 en Revise the outline organization of your annotated bibliography wheneveradvances in your understanding 0 a particu ar field demand it to fill holes in the knowledge represented by our bibliography by fin in relevant articles If an artice ound to fil a hole then you may have found a good idea for a thesis project Use the outline structure of the annotated bibliography to help organize the paragraph flow of the introduction section of your proposal Psvc iz summaries Epistemology From academicbrooklyn cuny edubismr virmalmlossaz htm The study of what is meant by quotknowledgequot What does it mean to quotknowquot something as opposed to merely having an opinion This issue has been at the core of Western philosophy since before Socrates since until it has been answered all other questions become unsolvable Psvc iz summaries Methods of Obtaining Knowledge Peirce 1877 Authority faith a trusted authority tells you what is e of false Tenacity sticking your head in the sand don39t consider or seek new knowledge believe only what A priori pragmatism believe what seems obvious sonable based on casual observation and common sense Scientific method Psvc iz summaries Features of Scientific Method Rational based on logic Empirical based on data Testable rational theories and hypotheses are testable Parsimonious the simplest explanation is most likely true General theories should account for broad phenomena Tentative explanations are never proved they may always be improved skeptical Rigoroust Evaluated replicate replicate replicate Selfcorrecting theories that are disproven are refined or abandoned Psvc iz summaries Theories and Hypothesis Testing What do theories do What purpose do they serve Understa1d39ng Prediction Organizing a1d Interpreting Rants Generat39ng Reseach What makes a theory a good theory Broad explmamr power Defines logical links between variables Predicts novel events recisely enough to be tesmd oonf rmed Predicts nonrevents Precisely enoug1 m be msmd d39scon rmed Parsimony induction Deduction Psvc iz summaries To Prove or Disprove That is the Question I Conditional Reasoning and the logic of falsification I39 I Theories Predict Data I Conf rmational Shategy h ying m prove a theory If theory A is correct then I will observe pattern of data Aquot I Discon rmational Strategy If th is correct n I will not observe pattern of data Bquot I These are statemenis of conditional reasoning rsvcm Research Methods To Prove or Disprove That is the Question I Conditional Reasoning The Propositional Calculus I Two premises and a conclusion Premise 1 If ltantecedentgt then ltconsequentgt Premise 2 Affirmdeny ltantecedentconsequentgt Conclusion Therefore ltconsequentantecedentgt I Four Possibilities for Premise 2 At m Awmoede Dewy Anteoedent Aff rm Consequent Dewy Comequewt rsvcm Research Methods To Prove or Disprove That is the Question I Confirmational Reasoning I remise If ltheoryA 395 mrrectgt men ltpattern Dfdata A Willbe observedgt enise 2 Conclusion dweory A ls cnrrett hererore data A will be observed Valid but polntless DA dweory A is l39nmrrett therefore data A will notbe observed lnvalld and pontiess A served hererore dweory A ls corth Invalid but ten used DC data A not observed hererore dweory A 395 hcnrrect valid but only if Dbse39vah39ons are ethaiAS Ve accwthy 0 19 null rsvcm Research Methods To Prove or Disprove That is the Question I Disoonfirmational Reasoning I Ise 1 1r ltdweory A c correttgt men ltpattern Dfdata B Wl39IInlthe observedgt Premise 2 Conclusion AA 39 mrrect hererore data B wlll notbe observed Valid but pondees DA theory A c n orrect mererore data B will be observed Invalid and poinliess Ac data B not observed mererore theory A is turned Invalid DC d ata B observed hererore heoryA a hmrrect valid most scientrrcally use ll rsvcm Research Methods Confirmation and Disconfirmation ofTheories Summary Con rmation Poor t theory correct thew observation will oocu observation occurs gt Support but not proof I Observation does not occur gt 39qaroof N0 D39soonf39rmation 0K t theory correct thew observation will not occur setyation does not occur gt SJpport but not proo Observation does oocur gt isproof strong Inteewoe BEST rsvcm Research Methods Strong Inference Platt 1964 I Science is fundamentally based on disconfirmation Popper I Theories are not evaluated in isolation rather they compete with one another relativism I Critical Experimenis resulis will disconfirm one or more theory theories while confirming one or more alternative theories I Disconfirmed theories are discarded or revised like logical branches pruned from the tree I of understanding 39n which only one branch represenis truth rsvcm Research Methods Next Time Topic More on Scientific Explanation measurement scales and descriptive stats Be sure to Read the assigned readings Howell chapters 1 amp 2 Identify a research area for your inclass posal Start searching and reading the scientific literature for your proposal stcaiz ResuvchMelhads PSYC512 Research Methods Lecture 5 Outline Questions about material covered in Lecture 4 Scientific Method Proof and disproof Strong Inference Brian P DYre Issues in Measurement University of Idaho mm momma mm momma To Prove or Disprove That is the Question Conditional Reasoning The Propositional Calculus Two premises and a conclusion Premise 1 If ltantecedentgt then ltconsequentgt Premise 2 Affirmdeny ltantecedentconsequentgt Conclusion Therefore ltconsequentantecedentgt Four Possibilities for Premise 2 Aff rm Anmoede Deny Anteoedent Aff rm Consequent Deny Comequent To Prove or Disprove That is the Question Confirmational Reasoning I Prem39 lse 1r lttneoryA is mrrectgt men ltpattern Dfdala A Willbe observedgt Pl ise 2 conclusion dieory A is cnrrett Mere Jre data A will be observed Valid but pointless DA dieory A is l39nmrrett tnererore data A will notbe observed Invalid and pontless A served diererore dieory A is correct Invalid but n used DC data A not observed mererore dieory A is hmrrect valid but only if observations are exhaustive ac rsvcm ResulchMelhads ceptng me null 7mm Mammal 7mm Mammal To prove or Disprove That is Confirmation and Disconfirmation the Question of Theories Summar con rmation Poor Disoonfirmational Reasoning t theory correct then observation Premise 1 W 00C If ltmeory A is correttgt men ltpattern ofdala B willnotbe observedgt 39 ztagfgo fm quot T S w 39t39 Premise 2 Conclusion Obsewa m does notocmr AA theory A is mrrect mererore data B will notbe observed d39qaroof N0 Valid butpoh s D39soonf39rmation 0K DA tneoryA is ncorrect mererore data B Willbe observed t theory correct then observation Invalid and pointless will not occur AC data B not observed Maefnre b lenry A 395 mrrett Invalid Observation does not ocmr gt DC data B observed diererore meory A is hmrrect valid most 31pmquot but quot0t 03900 scienti cally use ll Observation does occur gt disproof strong Inference BEST rsvcm ResulchMelhads Strong Inference Platt 1964 Science is fundamentally based on disconfirmation Popper mpete with one another relativ Critical Experimenis resulis will disconfirm one or more theory theories while confirming one or more alternative theories Disconfirmed theories are discarded or revised like logical branches pruned from the tree of understanding in which only one branch represenis truth Theories are not evaluated in isolation rather they co ism rsycm Research Mam Strong Inference Platt 1964 The Question m as 39n your own mind on hear39ng my scienti c explanation or theory put forwad What experiment oould disprove your hypotnes39squot or on nea39ng a scienth experiment dacribed What hypothesis does your experiment d39sprove quot Practic39ng explicit and formal analytical think39ng the notebook oonta39ning the logical trees alternative hypotheses md cmcial experimen 39nclude as an appendix m you annotated bbliography rsycm Research Mam Measurement What Makes Observation Systema ic uCareful planning of What will be observed How the observations will be made uWhen the observations will be made querational definitions translations of concepts stated in your hypothesis into the operations you will use to manipulate or measure that concept rsycm Research Mam Choosing Measures Research tradition e operant conditioning lever pressing eg cognition accuracy and reaction time eg sensation and perception discrimination accuracy e personality surveys inventories selfreports eory eg the psychophysical postulate discrimination accuracy e Serial vs parallel processes in visual search RT Availability of new techniques Availability ofequipment rsycm Research Mam Features of Measures Scale of Measurement Stevens 1946 Four types nominal ordinal interval and ratio Nominal scales set of mique cases es or categories with N0 ORDER valid operations ae nonparametric counting frequencies modes chi squae po39ntrbiserial correlation Ordinal scales drferent categories that cm be rmked along a cont39nuum moreor tnothowrru rnoreorla I valid operations ae nonrparamehic counting frequencies modes medians d1irsquae ranksorder correlation rsycm Research Mam Features of Measures Scale of Measurement Stevens 1946 Interval intervals of the scale are equal in magnitude valid operations parametric all mathematical era ions means an variances linear an non linear regression ttests ANOVA no fundamental zero no ratio statements allowed Ratio Like interval but also has a fundamental zero point allows ratio statemenis such as A is twice as much as Bquot Generally interval or ratio scales should be used if possible rsycm Research Mam Features of Measures Sensitivity Sensitivity measure must show changes in response to changes in the independent variable Range effects Ceiling effects variable reaches its highest po sible value and gels truncated test is too eas Floor effects variable reaches its lowest possible value and gels truncated test is too har stcaiz Research Mam Features of Measures Reliability the ability of a measure to produce consistent resulis when repeated measuremenis are take under identical condition Types of reliability precision physical measurement 1noise rgin of error sampling in surve interrater reliability observers viewing the ehavior Testretest parallel forrrs and splithalf reliabilities psychological tests stcaiz Research Mam Features of Measures Accuracy oes a measure produce resulis that agree with a known standard Accuracy vs Precision I Validit Measurement validity the extent to which your measure in eed measures what it is intended to measure Types Face validity Content validity Criterion related validity concurrent vs predictiv Construct validi Relationship between reliability and validity stcaiz Research Mam Next Time Topic descriptive statistics variables sampling and more on hypothesis testing Be sure to Read the assigned readings Howell chapters 34 Continue searching and reading the scientific literature for your proposal stcaiz Research Mam PSYC512 Research Methods Brian P Dyre University ofIdaho stcaiz Research Mam Lecture 11 Outline I Causation and Experimentation I Experimental research designs I React39v39 I Within vs I Condition Subject assignment I More on research design I Using 2 or more groups etween subjecls designs order39ng I Multifactor research using two or more independent variables Experimentation vs Quasiexperimentation stcaiz Research Mam Experimental Designs Manipulate one or more independent variables and observe effect on dependent variable Possible to achieve strong internal validity if extraneous variables are carefully oontrolled causation can be inferred Extraneous variables subject and environmental you aren39t interested in add error variance Differ n atmenls might be error variance rather than your man due to ipulations stcaiz Research Mam Experimental Research Reactivity I Reactivity and the Hawthorne effect I Demand characteristis cues in experiment that allow a subject to determine the experimenter39s purpose hypotheses or expectations Good ilbject role 5 produces expected effect I Failhfulrsubject role neuhal I Negativis crslbjeti role S sdmtages expected effeti I Countering demand characteristics I unobtrusive mealres or observations done in the eld Deception Withholding information stcaiz Research Mam Experimental Designs I Goal minimize the amount of error variance and ensure that it doesn39t correlate with your independent variables I How I Reduce error variance I Increase effectiveness variance of your IV by choosing more extreme treatmenls I Randomize error variance across groups through random assignment of sub39ects I Use inferential statistics to estimate the effecls of error varianc stcaiz Research Mam Features of Experimental Designs I Subject Assignment WithinSubjects repeated measures vs Between Subjects I Number of Independent Variables Single factor IV vs multiple factors IVs I Number of Dependent Variables Single DV vs multiple DVs multivariate stcaiz Research Mam Features of Experimental Designs Factors that determine optimal subject assignment If at all possible use a withinsubjects design Most efficientsrequ39res fewest ilbjects Statistically powerful ead1 slb39 ct acE as t ne39r own conhol elim39nates error variance due m ilbjeti vaiables PROBLEM Carrysover and order39ng eneas If significant carry over and ordering effecls are expected then use a betweensubjecls design Rmdomized Groups if ilbject variables ae not Mamed Groups r ilbject variables ae ex ected m may with the mealre some ague rm omiza on is still betmr rsvcm Wigwam Controlling for Carryover and Order Effects alance carryover and order effecB across treatments Randomization and Blocked Randomization Counterbalancing of N treatments omplete present each subject with a unique order and use every possible order requires N ordersSs Partial Latin Square present each subject a unique order carefully chosen from a subset of all possible orders rsvcm Wigwam Constructing a Latin Square for N treatmenB Randomly assign each treatment a number Five treatments A B C D E are assigned Determine First Order using 1 2 N 3 N1 4 N 2 A B E c D rsvcm Wigwam Constructing a Latin Square for N treatmenB Fill in N1 more orders by incrementing down and wrapping For odd N also use reverse orders N44Ss N510Ss ABDC ABECDDCEBA BCAD BCADEEDACB CDBA CDBEAAEBDC DACB DECABBACED EADBCCBDAE rsvcm Wigwam Controlling for Carryover and Order Effects Minimize carryover and order effects across treatments Practice Sessions Make the treatment order a betweensubjecs 39 d b in ependent varia le Creates a mixed design rsvcm Wigwam Controlling for Subject Variance in Betweensubjects Designs Random Assignment Ensures subject characteristics don39t oorrelate with treatments if you have enough subjects Matched Groups Distribute likeparticipanls to groups Subject attrition could be a problem Both methods attempt to equate the groups and treat subject variance as error variance rsvcm Wigwam Choosing the Number and Type of Independent Variables Single factor IV vs multiple factors IVs Single factor designs are simpler but more limited in scope Multiple factors allow for examining the synergistic effects of variables interactions Parametric vs metric Designs Parametric IV is quantitative ratio or interval scale Nonparametric IV is qualitative nominal or ordinal scale stcaiz Wigwam Experimental vs Quasi Experimental Research Designs Experimental Research random assignment of subjects to con 39 39 s Quasiexperimental assignment of subjects to groups base on a subject variablequoto measured attribute Treals a DV as an IV in the hope of establishing causality May be necessary in the context of field studies Examples age income testperformance Problems Confound39ng vaiabies may covary with sibject variable Regression m the mean stcaiz Wigwam Next Time Multi factor experimentation stcaiz Wigwam PSYC512 Research Methods Brian P Dyre University ofIdaho rsycstz Research Methods Lecture 4 Outline Questions about material covered in Lecture 3 Scientific Method Scope and assumptions Theories and hypothesis testing Proof and disproof Strong Inference rsycstz Research Methods Methods of Obtaining Knowledge review Peirce 1877 Authority faith Tenacity sticking your head in the sand A priori pragmatism Scientific method rsycstz Research Methods Features of Scientific Method Rational based on logic Empirical based on d ta Testable rational theories and hypotheses are testable Parsimonious the simplest explanation is most likely true General theories should account for broad phenomena Tentative explanations are never proved they m y always be improved skeptical Rigoroust Evaluated rep icate replicate replicate Selfcorrecting theories that are disproven refined or abandoned rsycstz Research Methods The Scope of Science The soope of science is limimd m questions that ae tractable using tlne scienti c method Objective reali eXI pbllosor by of materialism or physrcalrsrn e we can talk 330th rallty rndne lrdvs rstperson Reality is deterministic causality exis1s how else would scienti c modes make predictions Reality is sysmma cally observable uestio m these mmptions are beyond tlne soope of sctenoe Eg questions of Does God Exist or tlneoriesof Inmlligent Designquot are metaphysical not scientmc e prec39sion ofscien 39ic answers eories depends on tlne rec39sion of the uestion Vague questions lead m vague answers g why do ba things hwpm m good peoplequot o e 5 m e a o 5 y S o F rsycstz Research Methods What mdltes a theory a good theory Theories and Hypothesis Testing what do theories do What purpose do they serve Understandhg Prediction m Organizing and Interpreting Rants I Generatng Resea Ch ll lducuol l Deduction Broad explanamr power Defines logical links between vanables Predicts novel events recisely enough m be tesmd conf39rmed Predicts nonnevmts Precisely enougn m be msmd d39scon rmed Parsimony rsycstz Research Methods To Prove or Disprove That is the Question I Conditional Reasoning and the logic of falsification I39 I Theories Predict Data I Conf rmational Shategy h ying m prove a theory If theory A is correct then I will observe pattern of data Aquot I Discon rmational Strategy If th is correct n I will not observe pattern of data Bquot I These are statemenis of conditional reasoning rsvcm Research Methods To Prove or Disprove That is the Question I Conditional Reasoning The Propositional Calculus I Two premises and a conclusion Premise 1 If ltantecedentgt then ltconsequentgt Premise 2 Affirmdeny ltantecedentconsequentgt Conclusion Therefore ltconsequentantecedentgt I Four Possibilities for Premise 2 At m Awmoede Dewy Anteoedent Aff rm Consequent Dewy Comequewt rsvcm Research Methods To Prove or Disprove That is the Question I Confirmational Reasoning I remise If ltheoryA 395 mrrectgt men ltpattern Dfdata A Willbe observedgt enise 2 Conclusion dweory A ls cnrrett hererore data A will be observed Valid but polntless DA dweory A is l39nmrrett therefore data A will notbe observed lnvalld and pontiess A served hererore dweory A ls corth Invalid but ten used DC data A not observed hererore dweory A 395 hcnrrect valid but only if Dbse39vah39ons are ethaiAS Ve accwthy 0 19 null rsvcm Research Methods To Prove or Disprove That is the Question I Disoonfirmational Reasoning I Ise 1 1r ltdweory A c correttgt men ltpattern Dfdata B Wl39IInlthe observedgt Premise 2 Conclusion AA 39 mrrect hererore data B wlll notbe observed Valid but pondees DA theory A c n orrect mererore data B will be observed Invalid and poinliess Ac data B not observed mererore theory A is turned Invalid DC d ata B observed hererore heoryA a hmrrect valid most scientrrcally use ll rsvcm Research Methods Confirmation and Disconfirmation ofTheories Summary Con rmation Poor t theory correct thew observation will oocu observation occurs gt Support but not proof I Observation does not occur gt 39qaroof N0 D39soonf39rmation 0K t theory correct thew observation will not occur setyation does not occur gt SJpport but not proo Observation does oocur gt isproof strong Inteewoe BEST rsvcm Research Methods Strong Inference Platt 1964 I Science is fundamentally based on disconfirmation Popper I Theories are not evaluated in isolation rather they compete with one another relativism I Critical Experimenis resulis will disconfirm one or more theory theories while confirming one or more alternative theories I Disconfirmed theories are discarded or revised like logical branches pruned from the tree I of understanding 39n which only one branch represenis truth rsvcm Research Methods Strong Inference Platt 1964 The Question m adc 39n your own mind on hear39ng my scienti c explanation or theory put forwad 5 what experiment oould disprove your hypothes39 or on hea39ng a scienth experiment dacribed What hypothesis does your experiment d39sprove quot Practic39ng explicit and formal analytical lhink39ng the notebook conla39ning the logical trees alternative hypotheses md crucial experimm 39nclude as an appendix m you annotated bbliography rsycm Research Mum Next Time Topic Measurement scales descriptive stats variables and sampling Be sure to Read the assigned readings Howell chapters 13 Continue searching and reading the scientific literature for your proposal rsycm Research Mum PSYC512 Research Methods Brian P Dyre University ofIdaho rsycm Research Methods Lecture 10 Outline Exam Tuesday of Next Week Will cover all lecture material all material in Howell Chapters 15 broad concepts assumptions from Howell Chapters 611 Questions about material covered in Lecture 9 The Normal Distribution Testing Hypotheses Inferential Statistics rsycm Research Methods Hypothesis Testing Inferential Statistics All inferential statistics are evaluating this ratio Effect ood Vaianoe Test statistic quotWe Error bad Variance Example test statistis Chisquare t F These test statistics have known distributions that then allow us to estimate p the probability of a T pe I error inappropriately rejecting the null hypothesis Decision to reject null is made by com ring p to some generally accepted criterion forType I error probability at 05 rsycm Research Methods How is the probability of a Type I error p calculated It depends on Scaling properties of your dependent variable DV 39s interval or ratio parametric tesls DV is nominal or ordinal non parametric tests Research design Experimental test differences on measure etween nditions or groups 9 ttest ANOVA sign test Chisquare MannWhitne Correlational test relations between different measures 9 Pearson productmoment correlation pointbiserial correlation etc Mariner in which you phrase your hypotheses One tailed vs twotailed tests rsycm Research Methods Four Questions with subparB to Guide Your Choice of Inferential Test what are the scai39ng properties of my meailres or dependent varidles many meastres do 1 have minaI gthow mmy categories dichotomous 2 or nons tomous gt 2 IsAre my man39pulations or 39ndependent variaioiesqtaiitatve disaem camgorles or quantitative If qualitative how mmy levels Nom Often quantitative variables are man39pulamd as discr m categories How many man39puiat39ons factors do 1 have Are e factors manpuiated independently and exhaustively factoriai design Are the hypothaes directionai or not Is effectsize strength of relationship important to my hypoteses How u If no did10 rsycm Research Methods Examples rsycm Research Methods Next Time The eltaml stcmz ResuvchMeMads PSYC512 Research Methods Brian P Dyre University ofIdaho rsvcsiz Wigwam Lecture 19 Outline l Inferential Statistics I Testing for differences vs relationships I Analyzing frequencies l Analyzing differences between means rsvcsiz Wigwam Using Inferential Statistics I Which Statistic I The statistical decision tree Howell Figure 11 I Testing for relationships vs differences a false distinction I Relation 39 39 rength of relationship bet bles I Differences comparing different groups or treatmenls on some measurement I But what causes these differences The relationship between the independent variable defining the rea merit and ependent varia ble I Hence testing for differences is really testing the relationship between the IV and DV rsvcsiz Wigwam Analyzing Differences Between Treatments Nominal md crd39nal Frequen Success vs Failurequot 7 Binomial Distrbution and The Sign Test MJIt39ple camgories gt 2 Multinomial d39strbution md ch39rsquae D Data 2 eatments or groups attest Corrparlrlg two independent samples HW3 Corrparlrlg two correlated or paired sarrples HW4 More than 2 treatments or groups sANO More than 2 independent vaiables r rrultifacmrANOVAs st 2 or more dependent vaiables o repeated meamres r MANOV covariate gt ANCOVA r Relations between meail Correauon or Regression rsvcsiz Wigwam Analyzing Frequencies Howell Chapter 5 I Benoulli Trials series of hdepmde lt triab that rallt in one of two mutually exclusive outcomes Eg co39n ips gender of babies bom 39ncrease of decrease in a meaire after application of a treatment I The B39nomial Dishbution rsvcsiz Wigwam Analyzing Frequencies Howell Chapter 5 Usng the binomial distribution Mean nurroer of successes Vaiance in nurroer of success Alba Testing Hypomesesusng the binomial distribution The sign Test Hols typically a a 0 50750 cnance of success of falure out that doem tbave to be mecase H1lstyplcallyp7q Plu in values for N x p and q and DFX direch provides the re ability that me patten of data oou a result given the null ypotlnesis is true Sum me probabiliues 300 for all nurroer gt x to get the total prooaaility l tarlt of ndlng pgtgtlt rnpor The sign test takes into aocount direct on of differences but riot magnitude rsvcsiz Wigwam Analyzing Frequencies Analyzing Frequencies Howell Chapter 5 Howell Chapter 5 I What about multiple more than 2 possible outcomes 39 Multimmia39 distribumquot Usng the rnulunorntal dtsthpuuon Meal39le k Varlal lce ll l X Nka 11 sung Hypotheses usrng the rrulurlomal drsthpuuon Ho lstyplcall p pXk 11ltCH outoorne has the sarne chance but at doesn39t have to be the case HIlSWDlCleyW 17pXI 7 w Plug in values for Ngtlta1d 13 and pog gtltZ gt9 drrectty provldes the probablllty that thrs particular pattern of data oould result given the null Hypothesls ls true lVLlst surn the probablllues for all pattens that devl ate equal to or more to get the total probdolllty n urne consumngl stc lz newsman rsycm ResulchMethads Analyzing Frequencies Analyzing Frequencies Howell Chapter 6 Howell Chapter 6 I EasierAlternative to Multinomial distribution Using yzwith trult39ple dimensions contingency C Square if test bbs 39eqmiesth R is the number of camgories 39n 31mm 5quot Wm 0quot the dimension def ned by the e othe39 dimension rows of me le I Cgrrgp are computed value k is the rurrber of C is me quotLIme of camgo es in o x 0 Va ue 0 3 cat cries in the variable distribution with dfk1 a is 2 observed frequency 39 R CN 19 dlme loquot defned bY 16 I Is the total rumba of columns of the male I Expected frequencies for for 93C ng observations 0 1 0b d fr f the null hypothesis Eis the expecmd frequency Comp corrpumd vane of y Eamem gf equency or typically Nk where N 039 each MOW m vane of yzdistrbution with is the total number of iis the category 39ndex dfRn1Cn1 observations Eis the expecmd frequmcy for ead1 camgory rsycm ResulchMethads i a1d i ae category 39ndices rsycm ResulchMethads Analyzing Frequencies Howell Chapter 6 Z teStS ttesrs I Assumptions of the xi test I Each observation is independent I Inclusion of nonoccurrences p of populauon ts known I p of populauon ts est rnated as s t df N71 Corrpatn g 2 palred or correlated sanples Drtrerence scores I Df N 71 corrparng 2 rndependent sanples I 2 N1 N2 7 2 Unequal sarn le slzes heterogenerty of valence an pooled varlal39lces stc lz neurcrnurus rsycm ResulchMethads ANOVA FStatistic Used when compa39ng more mm 2 mems or 2 or more facmrs mmpu ms Homogeneity of vaimce Normality Independence of observations Between Goups compa39sons k rumber of mems compaed n rumber 0sz in goup Repeamd Measues Error term is 39nterac on of error wim le ilbject random vald rsvcm ResuvchMeMads Interpreting SPSS output rsvcm ResuvchMeMads PSYC512 Research Methods Brian P Dyre University ofIdaho stcaiz Research Mum Lecture 16 Outline Relational Research Observational Methods Assessing Reliability of observations Sampling stcaiz Research Mum Relational NonExperimental Research Observational Research Naturalistic Observation Archival Research Surveys selfreports All of these methods require measurement of behaviors directly viewed selfreported or previously cataloged stcaiz Research Mum Establishing Reliable Measures for Relational Research Nonexperimental research requires the definition of specific behavioral categories or recording unils Categories should be based on hypotheses informal observations literature search developed before the behavior is observed or the archival data is analyzed simple and focused on specific behaviors or archival te t con n exhaustive mutually exclusive independent stcaiz Research Mum Establishing Reliable Measures for Relational Research Behaviors within each category should be quantified using one or more of the following methods Frequency method number of times behavior occurs Duration method how long a behavior lasts Intervals method does behavior occur within discrete time intervals Behavior sequences keep track of order of behavior in addition to frequency stcaiz Research Mum Establishing Reliable Measures for Relational Research Problem Often behavior is complex and occurs too quickly to both observe and record at the same time Sampling methods Time alternate observing and recording periods Individual observe and record only one individual at a time Event observe and record only one behavior at a time Recording devices video audio stcaiz Research Mum Establishing Reliable Measures for Relational Research Problem any single observer or content analyzer might be biased or their observations might be idiosyncratic or unreliable Solution use multiple observers and quantify their differences by computing interraterreliability rsyom ResulchMefhads Establishing Reliable Measures for NonExperimental Research Slan39stical melhods for oorrpub39ng inmrramr reliability Percent agreement 100 NagreementsNobserva ons Cohen s Kappa K Po epc 1 7 Pc P0 is me aemal agreement and Pc is me agreement you would ect by cnance Confusion Matrix obseryer anulv luvan angry 15 i 13 13 26 P0 cell11cell22N 10 8 25 Pc row1col1 row2col2NN 1513 11132525 stc5 2 Resulch Methods Establishing Reliable Measures for NonExperimental Research Statistical methods for computing interrater reliability Pearson39s productmoment correlation Observer bias Blind observers Objective vs interpretive recording rsyom ResulchMefhads Sampling Why sample We cannot usually measure the entire population of interest so we mu sample of the population Goal of Sampling to be able to generalize to everyone in the population of interest sample must represent the population to insure external validity Terms st rely on measuring a population any group with size greater than 1 element one member of a sample eg person family city country etc strata subgroup of sample which is homogeneous with respect to some variable eg malefemale rsyom ResulchMefhads Random Sampling Techniques Simple random samp e l need an entire list or access to all elements of population draw sample using names in drum random number table etc given a bi enough sample it will be representative eac r of popu ation has an equal chance of being sample Systematic random sample short cut still need list of every element take every nth element where n pop sizesample slze pick first element randomly rsyom ResulchMefhads Stratified Sampling Techniques Stratmed Homogeneous Subgoup Sample Proporb39onal Sbab39red Sample sample elemenls are 39n me same proportion as they occur in me population allows 39nferenoes from sample slrara m population slrala inferenoes from ent39re sample to ent39re population roblem small slrala may not give enough detail Equal Slratmed Sample equal proportion ofsarrple oomes from ead1 slrala of population different size of slrala populations 7 hilres slability of sample from smaller slrala e sbala is equally represenlab39ve of ils target population allows corrpar39sons between slrara e 39ntemally valid LE views of polilical pa es 39n America rsyom ResulchMefhads Other Sampling Techniques Purposeful Sample identify cluster of sample that is representative of entire population with respect to the variable of interest randomly select from cluster Incidental convenience Sample sample from convenient or available population eg subject pool most psychological research does this surveys sample only people in phonebook d phone external validity is limIte stcmz Research Mam A Few Observations on Surveys really no pun intended Good source of information Dillman 19 and Telephone Surveys The Total Desrg Method New York Wiley and Sons stcmz Research Maw 78 Mail 7 Next Time Article Discussions stcmz Research Mam PSYC512 Research Methods Brian P Dyre University ofIdaho rsvcm Research Methads Lecture 8 Outline Questions about material covered in Lecture 7 Measures Reliability Precision and Validity Defining Variables and Research Designs Describing Data Testing Hypotheses Inferential Statistics rsvcm Research Methads Understanding Variability Visualizing Variability Distributions of Frequency and the Histogram Histograrrs used to represent 5m Hague I What iS variability frequencies of data In different D D classes or categories i D 2 u How is variability related to probability 6 3 u gt 4 3 g l 5 i g E E I 7 4 a 2 nizaoaevaam 9 B Grade in 1 wow summing mam summing Displaying Histograms Stem and Leaf Plots Distributions of Probability Density Stern and Leaf plots are used to display histograms graphically on their side using only typed characters Similar to frequency Stern hypothetical histogram for IQ hismgram eWEDt quotaxis 5 now represents 3 7 35668 probability density 3 3 s 012234445555667777889 mass rather than 3 9 00011233333334445566667889999 frequency E II 10 0111333334444445566677777888899 g 11 0001 122233444566777899 i m 12 Probability density H n Fl 13 FrequencyN u i 2 3 a 5 a 7 a a in Grade rsvcm Research Methads rsvcm Research Methads F mhahililvDensW Some Types of Probability Density Distributions Normal Gaussian Gamma m2 15 g m i 5 m2 395 3 min nus E DUE mm 5 395 1 um mm m u 5 is 2 u 5 is 2n Dela Dela rsvcm Research Mamas Describing Distributions Estimators and Parameters Sample statistics estimate population parameters eg Sample mean M or estimate the mean of a Sample variance seestimates the variance of a population 0 2 Properties of Estimators Sufficiency extent to which statistic uses all information observations available in sample Unbiasedness extent to which expected value of statistic approaches population value with increased sampling Efficiency tightmssiofielustamofssample statistis Measures of the Center of a Population or Sample Measrres of center represent the general magnitude of scores Mode most frequent score Median the middle score ofa1 ordered list Mean average where x repraenE a of observations I Which meailres He the most ilf cimt thimed Ef cimt Resistant rsvcm Research Mamas Measures of the Spread of a Population or Sample Measrres of spread are used m asess me oonsismncy of scores 39n a dishbution Range max score 7 min score Inmrquatile range sooreQ3 sscoreQ1 Variance d5 z and smdard deviation gs where Xis a vecmr of data M is me mean of me popu 39 n is me mean obsev Ions I Whid1 mealres He the most Ef cient Unbiased Ef cient Resisbmt rsvcm Research Mamas Standard Deviation Standard Deviation Q sqrtvariance where X is the ata m is the mean of the data and N is the total number ns of observatio Remembering how to compute variance the mean of the squares square of the meansquot rsvcm Research Mamas Dacribing Distributions Parametrically Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment Mean location of distribution on xaxis Second Moment Variance dispersion of distribution Third Moment Skewness symmetry of distribution Fourth Moment Kurtosis degree of peakedness rsvcm Research Mamas Testing Hypotheses I Hypothesis testing is the process by which hypothetical relationships between intervening variables are assessed I Hypotheses are always tested relative to one ther or to a null hypothesis 5 I Comparing groups I Assessm erformance interventions I Assessing relationships between variables rsvcm Research new Null Hypothesis Testing and Inferential Statistics actually exists H1 the experimentaloralternative ypothesis I 2 possible decisions when looking at the data I Conclude that a relationship exists Q39eject the null hypothesis H09 DISCONFIRMATION I Conclude that no relationship exists do not reject the null hypothesis 9 CONFIRMATION NO rsvcm Research new Null Hypothesis Testing and Inferential Statistics 2 realities by Tme State ofthe World 2 decisions form a 2 x 2 Reject HL7 matnx of 4 possibilities D e cision H a conclude there is NOT an effect rsvcm Research new Hypothesis Testing Probability and Statistics Problem How do we distinguid1 real diferenoes or relationsnbs from mealremmt no39se Probability and statistks may be used to ages desa39pu39ve statistits or oompae inferential statistits the relative magnilude of differ 39 39 Effect treatment Varia1ce Variability due m relationship between vaiables or effect of different leres of independent vaiable treatments I Good varia1ce that we want m maximize ent types of Val I I I Error Vaiance I Variability 39n measure due m factors othe tha1 the treatment I Bad vaia1ce that we wa1t to m39nimize rsvcm Research new Hypothesis Testing Inferential Statistics I All inferential statistics are evaluating this ratio Effect good Valanoe Test statistic Error bad Variance I Example test statistis Chisquare t F I These test statistics have known distributions that then w us to es imate p e r0 bili o a pe I error inappropriately rejecting the null hypothesis I Decision to reject null is made by comparing p to some generally accepted criterion forType I error probability 05 rsvcm Research new Null Hypothesis Testing and s Inferential Statistic 1 Population I Why might we observe a E difference between two gr groups if no difference 5 from the same H 2 samples population Each sample may have a unique mean due to sampling error Frequency kl k rsvcm Research new NullHypothesis Testing and Inferential Sta is i s 2 Populations How does this change if a difference actually exists between my groups Each sample has a unique mean that 1 1 represenls oth sampling error and the differences between the 2 populations Frequency Frequency stc iz ResuichMefhads How is pcalculated It depends on the scaling properties of your dependent variable DV 39s interval or ratio parametric tesls DV is nominal or ordinal non parametric tests esearch design Experimental 7 test dm erences on meailre between conditions or groups gt ttest ANOVA sign test Mann W itney I Correlational itest relations between differmt mealres 9 Pearson productrmoment correlation po39ntrbiserial correlation em the manner in which you phrase your hypotheses One tailed vs twotailed tests stc iz ResuichMefhads Next Time Topic Normality Probability NuB and Bolts of Testing Hypotheses Be sure to Read the assigned readings Howell chapters 67 Continue searching and reading the scientific literature for your proposal stc iz ResuichMefhads PSYC512 Research Methods Brian P Dyre University ofIdaho stcalz Research Methods Lecture 13 Outline Review of Lectures 1112 Causation and reactivity Within vs between subjects designs More on research design Multifactor research using two or more es independent varlabl An example of multi factor research Experimentation vs Quasiexperimentation stcalz Research Methods Factorial Research Designs Used to assess the effecB of 2 or more independent variables factors on your dependent variab e Using multiple IVs in one experiment is more economical particularly for withinSS provides more information g lvain effecls of each IV separate effects of each Interaction between the IVs synergism effect of one variable changes across the levels of the other variable st c512 Research Methods Factorial Research Designs Possible Outcomes of a 2 x2 Design Noeffects Ma39nEfectACnly gt gt 5152 gt 9 5152 D 9 A1 A2 A1 A2 Factor A Factor A stcalz Research Methods Ma39n Effect 5 only 5z 1 5 A1 A2 Factor A Factorial Research Designs Possible Outcomes of a 2 x2 Design Marn Effects Main Effects Main Effect Main Effect forA ahd5 forA ath or A only for 5 only No Interaction Interaction Interaction Interaction or 39 disordinal ordinal B2 52 B2 51 B2 gt f a a D D Bl El Al A2 Al A2 Al A2 Al A2 Factor A Factor A Factor A Factor A IMPORTANT always interpret the highest order effect or interactj on stcalz Research Methods An experiment on Example Factorial Experiment Perceiving Heading steering control vs riding and pointing stcalz Research Methods Example Factorial Experiment Perceiving Heading 1V1 observer39s task Steering control Steer so that you appear to be moving straight ahead Pointing Point in the direction towards which you perceive yourself to be moving while r39 ingquot 1V2 simulated velocity of observer movement Three levels 50 100 and 200 msquot Dependent Variable measure RMS rootmean squared heading error sqrt2errorN stc iz Research Methods Results Signi cant Effects 39 Task Main Effect 39 Interacti on between task and velocity IMPORTANT ALWAYS INTERPRET HIGHEST ORDER EFFECT w Puinting Task VaWnCuntrui Task RMS Heading Error deg r tun iEEI Velocity ms stc iz Research Methods Specialized Research Designs Combining betweensubjecls and withinsubjecls factors in research design mixed designs Combining experimental and correlational designs Analysis of covariance or ANCOVA QuasiExperimental Designs Pretestposttest designs Developmental designs Longitudinal or crosssectional stc iz Research Methods Combining BetweenSubjects and Within Subjects Designs The Mixed Design Also known as split plotquot Groups of sub ecls each receive a unique level of s t e tween S variables and all levels of the within Ss variables stc iz Research Methods Mixed Threefactor Design Adding Presentation Order as a Factor to Account for Differential Carryover GiouvAVwtcomml enopo VoiMinV etnope ninth Variable between em a New Group A contro1 then pointing Group B pointing then control constant nesntno Ermr deg Found3 wayinteraction that indicates differential ca over asymmetrical transfer so too tso zoo o so too tso zoo Velamy n is Velautv ms Taski stc iz Research Methods Combining BetweenSubjects and Within Subjects Designs Exarrpie ofa Mixed Design Tadlt orde manipulated between SS Group A oontroi then poinhng Group B point39ng than control enmsnnennt GNIDA mug isomo mng Gmlvl suiilgcolwi 5 u consent neantno Ermr deg VelamtvUiiS Velautv ms Ta sk i Task 2 stc iz Research Methods k I o o so too tso zoo o so too tso zoo Combining Experimental and Correlational Designs Covariams in experimental designs Measire your sibiecns on a coyariate a vaiable tlnat you believe may be correlated wilin yo r dependent variable aiams add error vaianoe and might Mealr39ng tlne covariam allows you m use correlational statistical mdmiques in your analys39s e g malysis of Covariance or ANCOVA m sibtract out tine error variance associated witln tine oovariam tinereby 39ncreas39ng tine statistical power of your experiment Example measu39ng IQ 39n a lean39ng experiment rsycm summaries Combining Experimental and Correlational Designs Quasiindependent variable in experimental designs Quasi mea kind of but not reallyquot Similar to including a covariate exce pt rement of covariate is used to assign Ss to ns Covariate is thus treated as an quasiindependent variable Quasiindependent variables are referred to as quasi because they cannot be manipulated they are essentially dependent variables measures that are treated as independent variables in the experimental design and analysis rs vow Resealch Methads Quasiexperimental Designs Quasiexperimental designs are those in which only quasi independent variables are used Time series vs pretestposttest desig s Time series Measure behavior several times prior to and following a treatment time series design or change in your quasiindependent variable interrupted time series design Pretestposttest Measure behavior once prio to once following the change in your independent variable rsycm summaries Quasi experimental Designs Equivalent time samples design Timeseries design especially useful for treatmenis with transient effect Repeatedly measure behavior following multiple applications and withdrawals of the treatment Nonequivalent control group des39gns helps control fo r history confounds which should affect both groups equally rsycm summaries Developmental Dsigns Used m mess dnanges 39n belnavior relamd to a person s chronological e serves as a qias39rindependent vaiable Crosssectional desigm s m ltaneously mst sibiects Eigned m two or more age goups Generational effects can confound tlne age variable Lorgitud39nal designs 5 m at est a s39ngle group of sibjects over time Controls for generational effectsJout may still limit external validity May be confounded by hismry mortality andor multiple observation e Cohortnsequential design Combines longitudinal and crosssec onal designs by measiring multip age groups over 39me 39 in a ws evauation of generational or hismrical oonfoun s stcm 2 Resealch Methads Next Time More on experimentation Smalln designs rsycm summaries PSYC512 Research Methods Brian P Dyre University ofIdaho stc iz Research Mums Lecture 7 Outline Questions about material covered in Lecture 6 Measures scales and sensitivity More on Measurement Reliability Precision and Validity Hypothesis testing and Variables Variables and Research Design Defining Variables stc iz Research Mums Features of Measures Reliability The ability of a measure to produce consistent results when re eated measuremenls are taken uri er identical conditions Types precision physical measurement 1noise margin of error sampling in surve s interrater reliability observers viewing the same behavior Testretest parallel fame and splithalf reliabilities psychological tests stc iz Research Mums Other Features of Measures Accuracy does a measure produce resulls that agree with a known standard Accuracy vs Precision Validity Measurement validity the extent to which your measure indeed measures what it is intended to measure Types Face validity Content validity Criterion related validity concurrent vs predictive Construct validi Relationship between reliability and validity stc iz Research Mums Hypothesis Testing Variables Hypoihes39s msting is the prooess by whid1 hypothetical reiab39onships between vaiabies sorneih39ng that varies in quantity or quality ae assaed the reiau39onships ae deduced from one or more theories Types of vaiabies Dependentvariable gt measvre atlo Extraneous variable gt not pertinent m hypothaes Confound39ng variable gt extraneous vaiabie thatcovaies with your mm39pulamd vaiabie typically we by m ooniroi mae to elim39nate the covariance Inmrvening variable gt theoretical oonsiruct of 39nterest that is not direcuy observable eg group cohesivena mental kload stc iz Research Mums Variables and Research esigns Relationships can be hypothesized between Multiple dependent measures 9 correlational research design presence or absence of a relation between the variables can be tested but not causality Manipulated independent variables and some measure 9 experimental design with proper control of confounding variables eg random assignment to experimental treatment groups causality may be inferred stc iz Research Mums Defining Variables Defining Variables Converging Operationism Operations or Network Specification Operationism psychological concepts are equivalent to the operations manipulations or to define a concept not just one measures used to define those concepts Operations can converge to scientifically isolate I Hunger the state produced by food intervening variables through a process of converging deprivation operations Garner Hake amp Eriksen 1956 only observable operations are included in selective influence experimental manipulations theoretical or hypothetical Statements affect particular intervening variables but not others I You cannot separate the concept from its convergence different operations can be used to Multiple operations or a set of operations can be used operations cannot generalize concept has map39pulate quot measure a mm quot 39quottervemmg variable or psychological construct no external validity stc iz mammals stc iz mammals Network S ecification of Converging Operations p Meaning Example The phenomenon of Perceptual Defense Psychological Concepis are defined by their relations Gamer Hake amp Eriksequot 1955 with other concepis rather than a unitary operational Two Possibilities definition perceptual discrimination of vulgar words takes Introduction and Discussion sections of papers describe longer the relationships of our variables to all other relevant 39 responding With a 13993quot word takes l0quot 9quot variables and concepts what G H amp E call assumed Operationist perception isthe discrimination response operations on we can tell Wh39ch Method and results sections describe the specific Converging operations add a second orthogonal t operation exchange the vulgar and neutral response convergmg Opera Ions we use mappings stc iz mammals stc iz mammals Construct Validity Testing Hypotheses The soundness of our operations do they manipulate or Hypothesis testing is the process by which hypothetical measure the intervening variable that they are intended relationships between inferveningvariables are assessed to maniPUIEte 0quot measum Hypotheses are always tested relative to oneanother or Types Campbell amp Fiske 1959 to a null hypothesis Discriminant validation operation should not affect or Examples correlate with operations on other Intervening Comparing Groups Var39ables Assessing Performance Interventions Convergent validation operation should affect or I Assessing Relationships between variables correlate with other operations on the same Int 39 ervening variabe Problem Measurement Noise rsvcm Research Methads rsvcm Research Methads Hypothesis Testing Probability and Statistics Used to assess variability In a me sure Effect treatment Variance independent variable trea menls Goodquotvariance that we want to maximize r Variance Variability in measure due to factors other than the treatment Bad variance that we want to minimize Probability and Statistics are sim Iy tools used to assess descriptive statistics an compare inferential statistis these sources of variability stcaiz ResuvchMefhads Visualizing Variability Distributions of Frequency and the Histogram Histograrrs used to represent 5m Heguem frequencies of data in different D D classes or categories 1 D 2 u a 3 u gt 4 3 g l 5 l E E E 1 7 4 E 2 nizaoae7sam 9 B Grade in l stcaiz ResuvchMefhads Displaying Histograms Stem and Leaf Plots Stern and Leaf plots are used to display histograms graphically on their side using only typed characters tem hypothetical histogram for IQ 6 7 35668 8 012234445555667777 9 00011233333334445566667889999 10 0111333334444445566677777888899 11 01111122233444566777899 12 13 stcaiz ResuvchMefhads Distributions of Probability Density Similar to frequency histogram except yaxis now represents yDensily probability density mass rather than frequency 3 1 E m Probability density H n l39l FrequencyN U12 soae 785m Grade mam ResearchMelhads Frnnamiw DensiW Some Types of Distributions Normal Gamma w W a m l g nu mm D um Bus 3 um mm 3 nus 5 um um m u 5 15 2 u 5 15 2D stcaiz ResuvchMefhads Measures of the Center of a Distribution Measures of center represent the general magnitude of scores in a distribution Mode most frequent score Median the middle score of an ordered distribution Mean average where X is the data and N is the total number of observations stcaiz ResuvchMefhads Measures of the Spread of a Distribution Measures of spread are used to assess the consistency of scores in a distribution g maxs re min soore Interquartile range scoreQ3 scoreQ1 Variance a and standard deviation 7 where X is the data m is the mean of the data and N is the total number of observations rsvcm murmur More on Variance Standard Deviation Q sqrtvariance ere X is the data m is the mean of the data and N is the total number of observations Why N instead of Nl Populations vs Samples Remembering how to compute variance the mean of the squares square of the meansquot rsvcm murmur Dacribing Distributions Parametrically Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment Mean location of distribution on xaxis Second Moment Variance dispersion of distribution Third Moment skewness symmetry of distribution Fourth Moment Kurtosis degree of peakedness rsvcm murmur Estimators Sanpie siat39stlcs estimatepopulation paameters Mem M or vs M rim vs Properties of Btimamrs Sufficiency uses all 39nformau39on 39n sample mean arld vaianoe ae svi cient mode and range ae not Unbiasedness expected value approad1es real value with increased samprng Efficiency lighma of clusmr of sample slapstlcs relau39ve to me populau39on paamemr Res39sbmoe infllence of outliers on sample shtis c rsvcm murmur Next Time Topic Research Designs and Inferential Statistics Be sure to Read the assigned readings Howell chapters 67 Continue searching and reading the scientific literature for your proposal rsvcm murmur PSYC512 Research Methods Brian P Dyre University ofIdaho stcsiz Research Methads Lecture 12 Outline Within vs between subjects designs Condition ordering Subject assignment More on research design Using 2 or more groups Multifactor research using two or more independent variables Experimentation vs Quasiexperimentation stcsiz Research Methads Features of Experimental Designs Withinsubjecls design Most efficientrequires fewest subjecls Statistically powerful each subject acls as their own control eliminates error variance due to subject variables PROBLEM Carry over and ordering effects Betweensubjecls design Randomized Groups if subject variables are not expected to covary with the measure Matched Groups if subject variables are expected to oova with the measure some argue randomization is still bet r 7mm re 2 Research Methads Controlling for Carryover and Order Effects Balance carryover and order effecB across treatments Randomization and Blocked Randomization Counterbalancing of N treatments Complete present each subject with a unique order and use every possible order requires N ordersSs Partial Latin Square present each subject a unique order carefully chosen from a subset of all possible orders stcsiz Research Methads Constructing a Latin Square for N treatmenB Randomly assign each treatment a number Five treatments A B C D E are assigned Determine First Order using 1 2 N 3 N1 4 N 2 A B E C D stcsiz Research Methads Constructing a Latin Square for N treatmenB wrapping 39 For odd N also use reverse orde N44Ss N5105s Fill in N1 more orders by incrementing down and rs ABDC ABECDDCEBA BCAD BCADEEDACB CDBA CDBEAAEBDC DACB DECABBACED EADBCCBDAE stcsiz Research Methads Controlling for Carryover and Order Effects Minimize carryover and order effects across treatments Practice Sessions Make the treatment order a betweensubjecB independent variable Creates a mixed design stc512 Research Methods Controlling for Subject Variance in Betweensubjects Designs Random Assignment Ensures subject characteristics don39t correlate with treatments if you have enough subjects Matched Groups Assess participanis on one or more characteristics that might correlate with the DV Distribute likeparticipanis to groups Both methods attempt to equate the groups and treat subject variance as error variance stc512 Research Methods Choosing the Number and Type of Independent Variables Single factor IV vs multiple factors IVs Single factor designs are simpler but more limited in scope Multiple factors allow for examining the synergistic effects of variables interactions Parametric vs Nonpara i Designs Parametric IV is quantitative ratio or interval scale Nonparametric IV is qualitative nominal or ordinal scale stc512 Research Methods Factorial Research Designs Used to assess the effecB of 2 or more independent variables factors on your dependent variable Using multiple IVs in one experiment is more economical particularly for withinSs provides more information Kgin effecis of each IV separate effects of each Interaction between the IVs synergism effect of one variable changes across the levels of the 39 ble other varla 75v 512 Research Methods Factorial Research Designs Possible Outcomes of a 2 X2 Design No effects Ma39n EfectA Only Ma39n Effect B Only 5152 52 gt gt gt 9 5152 D D 51 A1 A2 A1 A2 A1 A2 FactorA FactorA Factor A stc512 Research Methods Factorial Research Designs Possible Outcomes of a 2 X2 Des1gn Main Effects Main Effects Main Effect Main Effect forA andB forA andB for for 5 only No Interaction Interaction Interaction Interaction ordi a1 disordinal ordinal B2 51 52 gt wg 41 a 51 A1 A A1 A A1 A A1 A Factor A Factor A Factor A Factor A IMPORTANT always interpret the highest order effect or interacti on stc512 Research Methods Example Factorial Experiment Perceiving Heading An experiment on perceiving the heading direction during steering control vs riding and pointing rsvcm Research Methods Example Factorial Experiment Perceiving Heading 1V1 observer39s task Steering control Steer so that you appear to be moving straight ahead Pointing Point in the direction towards which you perceive yourself to be moving while r39 ingquot 1V2 simulated velocity of observer movement Three levels 50 100 and 200 ms1 Dependent Variable measure RMS rootmean squared heading error sqrt2errorN rsvcm Research Methods Results 39 Task Main Effect Interaction between task and velocity Puinting Task VawrCuntrul Task iEIEI iEEI Velocity ms i IMPORTANT ALWAYS INTERPRET HIGHEST ORDER EFFECT RMS Heading Error deg 2mm 2 n rsvcm Research Methods Specialized Research Designs research design mixed desi Combining experimental and correlational designs Analysis of covariance or ANCOVA QuasiExperimental Designs Pretestposttest design Combining betweensubjecls and withinsubjecls factors in ns s Developmental designs Longitudinal or crosssectional rsvcm Research Methods Combining BetweenSubjects and Within Subjects Designs The Mixed Design Also known as split plot Groups of subjects each receive a unique level of the between Ss variables and all levels of the within Ss variables rsvcm Research Methods Mixed Threefactor Design Adding Presentation Order as a Factor to Account for Differential Carryover earpamema mpa mm Vanable between mwa mm W a mama Group A contro1 then pointing Group B pointing tnen contro1 Constant Heading Ermr deg Found 3 way interaction that indicates 5n inn 15D mu H 5B inn 15D 2cm differential carryover WWWs Veiww Ms llt asymmetrical transfer rsvcm Research Methods Combining BetweenSubjects and Within Subjects Designs NeXt Tlmeu Quasiexperimental designs 9 m Smalln designs 5 u an mu 15m 2m u an mu 15m 2m Example of a Mixed Design then pointing Group B pointing then oontrol D U l a 9 3 a z 2 2 7r 1 m 1g o 3 2 F quot g w E 1 i CnnsiamHeamngErmrmeg 39 3393 39quotteracmquot VeiaulVUViS VeinWWs quot quotate Task 1 TaskZ differential carryover effect stcmz summing stcmz summing asymmetrical PSYC512 Research Methods Brian P Dyre University ofIdaho stc iz Research Melhads Lecture 2 Outline Class introductions Questions about syllabus or website Go over problem set assigned last time Overview of the Research Process Searching the literature Reading the literature stc512 Research Melhads The Research Process Develop a research idea and hypothesis Develop idea into a testable h thesis 3 Choose a research design Experimental Correlational Quasirexperimental Obtain subjects sampling ethical and practical considerations The is four steps form the bass of your research pro 035 for W class ad a mess proposa should you decide w my ete one Conduct the study Analyze the results Report your resulls Conferences Journals Repeat Steps 17 forever stc iz Research Melhads Searching the Literature Electronic Resources The UI Library electronic databases Social Science Citation Index SSCI available online Google Scholar beta Hardcopy Resources the old traditional wayquot Textbooks 9 identify seminal research papers use SCI to search fonvard in time for more recent works that were influenced Reference sections of papers 9 trace back in time to previous research that influenced the current work stc iz Research Melhads Searching the Literature General Strategy Initially obtain a broad overview of a particular area reading review papers or books and seminal works rather than tightly focused empirical reports 9 helps to organize understanding and identify key issues Once broad overview of a research area has been obtained delve into more tightly focused empirical re rovides more nuts and boltsquot methodological information and data stc iz Research Melhads Organizing Reading and Most Importantly Remembering the Literature Never read passively Always have a goal in mind obtaining some particular piece or type of information Observe your circadian rhythms for periods of s and take advanta t Never read without taking notes to include in your annotated bibliogra hy Plumb the reference section for further readings stc iz Research Melhads Writing an Annotated Bibliography As you read cream a1 outline of the general area of 39nqu39r tlat is organized by irrJJortant isles emp39rical rants theoretica stances hypotheses a1 questions a hebs m organize understmd39ng and identify key Eiles In your outl39ne hcude entries for every paper you have found that oonta39ns the full citation perth lncudlng the abstract for the most releymt pages and a hypetrl39nk m the full text if available electronically mine or on a local dHlt Sometimes you may WH1 m include references to a particula paper in rrult39ple places in your outl39ne Ead1 citation should be followed by detailed noms of key tales ra39sed by each paper eg hypothaes and rationale general method most important results and your own comments relation of work m other raead1 and your own remrch project relative lnportanoe of work followrup qJeshons rsycm ResealchMefhads Using Your Annotated Bibliography Think of your annotated bibliography as your memoryquot of the research in your field of interest update and refer to i 0 en Revise the outline organization of your annotated bibliography wheneveradvances in your understanding 0 a particu ar field demand it to fill holes in the knowledge represented by our bibliography by fin in relevant articles If an artice ound to fil a ole then you may have found a good idea for a thesis project Use the outline structure of the annotated bibliography to help organize the paragraph flow of the introduction section of your proposal rsycm ResealchMefhads Next Time Topic Scientific Explanation Be sure to Read the assigned readings Identify a research area for your inclass proposal rsycm ResealchMefhads PSYC512 Research Methods Brian P Dyre University ofIdaho steaiz Research Mum Lecture 17 Outline Lecture 16 Review Relational Research Sampling Research Ethics Writing Research Proposal The Art of Peer Review Addressing Reviewer39s CommenB steaiz Research Mum Ethics associated with Human SubjecB The unspoken subjectexperimenter contract Subject expectation is that experimenter will give clear instructions ensure safety amp warn of dangers inform subjecls of the nature of the experiment Experimenter expectation is that subjects will coopera be honest faithful In psychology subjecls frequently see experimenter t not as a scientist but rather as a therapls steaiz Research Mum Institutional Ethical Oversight Institutional Review Board research must be approved priorto conducting the research approval does not lesson responsibility of the experimenter Federal Review may also occur for projecB receiving federal funding eg NIH NIMH steaiz Research Mum Basic Ethical Principles Minimize harm to participanB Informed consent Freedom to withdraw Protection from harm Confidentiality Maximize benefit to science steaiz Research Mum Informed Consent Ss should be given a description of what they will do and possible problems and detrimental effects the fthe experiment Ss must sign a consent form that describes risks or discomforts and explicitly states that rticipation is volunt the subject may withdraw at any time without Dena v responses are confidential or anonymous Deception cover storyquot used to minimize reactivity to experimental procedures if used must provide a full debriefing steaiz Research Mum The Debriefing As a part of informed consent subjects should be told of the full nature of the study Especially important if study causes temporary detrimental effecB or uses deception Goal identify and remove misunderstandings Opportunity to collect additional data Opportunity to educate rsvcm Research Methods Freedom to Withdraw Ss must be allowed to withdraw from an experiment at any time Ss must KNOW they have the freedom to withdraw rsvcm Research Methods Protection from Harm Experiments that subject Ss to any risk require very strong justification Experimenters responsibility to anticipate risks and take measures to minimize risks rsvcm Research Methods Confidentiality Information given by or collected from Ss should be kept confidentia Sometimes confidentiality conflicls with protection from harm Easiest manner to insure confidentiality is to collect ta in a manner that makes the data anonymous Maintaining confidentiality Data security Keep information coding identity separate from the data itself rsvcm Research Methods Maximizing benefit to science Minimize waste by using pilot studies insure equipment and stimulus materials are appropriate and working correct eliminate bugs from data collection and data reduction analysis Make sure your measures are reliable rsvcm Research Methods Writing a Research Proposal Purpose of Research Proposal Presents a literature review that defines the concepts an intervening variables pertinent to a particular research question Presenls a specific research question with explicit hypotheses to be tested Develo a Ian for addressing that research question empirically including descriptions of Taget population and ilbjeti sanpl39ng The remrd1 des39gg 39ncuding ex licit def nition of 39ndependmt and e1de1t vaia les a1d how the stimuli and prooedures irrp ment thae vaiabies ects of stimuli and procedures that hep m contml for ex meous valables a1d confoundng varld es Types of analyses to be used Eradicth rains and how thae predictions relam to the ypothaes c512 Research Methods Research Proposals Global Concerns I Scientific Writing Style I Precision more important than entertainment but Research is part science and part advertising not only do you need to develop good ideas but you must be able to sell your idea s I Proposal is often the basis of first impression for the quality of the research pro39ect I Proposal must be clear on ALL levels of analysis Words Sentences Paragraphs OUTLINE Sections 2 rsvcsnz ResulchMethads Research Proposals Global Concerns l APA guidelines l Orderly expression of ideas organization I Smoothness of expression transitions l Economy of expression concise language Precision and clarity use scientific vocabulary jargon correctly and insure that all terms are defined the first time they are used rsvcsnz ResulchMethads Organization of a Research Proposal USE APA STYLE I Title Page title should specifically describe what the paper is about so that it is useful information for other researchers39 iterature searches stract essentially a minipaper for lit searches extremer CONCISE lt 150 words introduoe specific topic discuss variables etc present major results no statistics I discuss important conclusions rsvcsnz ResulchMethads Organization of a Research Proposal USE APA STYLE I Introd uction Demonstram knowledge of relevant reseacln I De ne 39ntervening variables and their relation m man39pulations and measuemenis u 39n previous reserach Present and jistify raead1 question and lnypotlneses Present and jistfy tine general metlnod m be used I Organization stat broad tlnen narow m your geneal purpose D39scuss only relevant research 39n a b ical ow Nea tine end provide an explicit smment of hypotheses and an overview of line general raead1 desigl rsvcsnz ResulchMethads Organization of a Research Proposal USE APA STYLE e hod I explicitly state how variables are manipulated efine in separate subsections I km I Desi n Stinuli pparatusMamriab cedures Resulls scales of IVs and DVs transformations of DVs list analyses you plan to perform summarize predictions rsvcsnz ResulchMethads Things to Avoid Brian s Proposal Pet Peeves Use of informal language e g you don t run ilbjecE you let subjects Ambiguous pronouns r you use tine word it make sire tine surrounding context makes tine meanng of line word it obvious otlnenuise avoid using it as in it Sex39st pronouns e vs sine 7 word sentean m avoid having to use thae if at al possble or use he or she Ve39b tense I IntrodJCtion Section discussion of ecific previous raeach gt past tense ideas or concepts tin are general for all time and not l39nked to past gt present tense tin fu r tense o an Pluralsingular mismamhes assive voice wor smtmces 39n active voice Supemous imprec39se language eg avoid vague adverbsi search for all words ending n 1y and oonside elim39nat39rlg ilnem style isles like that vs whid1 sycsnz Resulch Mam The Art of Peer Review Example review available online in lecture schedule Goal Assist author in improving the clarity and impact of the paper or proposal by offering specific constructivecriticism no name calling and complements where appropriate Stmcture Summary Major or general criticisms typically lt3 Minor specific criticisms any number specifically listed by page and line number stcmz ResuvchMefhads Addressing Reviewer s Comments Never blame a reviewer for a negative review Consider all criticism as constructivehthe reviewer is trying to Assume misunderstandings are your fault not the reviewers All misunderstandings occur because you did not write clearly enough The cover letter required with submission of revised proposal Specifically responds to the reviewers comments by describing specific changes made in the paper to address criticisms Presenting a rationale for why a reviewer39s criticism was not addressed Must be worded very diplomatically stcm a hummus 2 mm Presenting Research Timeallotted 20 minutes 9 strictly enforced se same general format as the written report introduction roughly 10 minutes method roughly 8 minutes Summary 2 minutes or so Given time allotted you cannot go into the same level of detail as yourwritten report Materials Talk from an outline of points you wish to make Visual Aids 9 power po39nt Elements of Style PRACTICEYOUR PRESENTATION Anticipate questions and how you will answer them stcm 2 Research Methads PSYC512 Research Methods Brian P Dyre University ofIdaho rsvcm Research Methods Lecture 5 Outline Questions about material covered in Lecture 5 Scientific Method Proof and disproofamp Strong Inference Operational definitions Issues in Measurement Me Scales of Measurement Variables Reliability and validity sampling rsvcm Research Methods Choosing Measures Research tradition operant conditioning lever pressing eg cognition accuracy an reaction time eg sensation and perception discrimination accuracy eg personality surveys inventories selfreports Theory eg the psychophysical postulate discrimination u Availability of new technlqu Availability ofequipment acc racy eg Serial vs parallel processes in visual search RT es rsvcm Research Methods Features of Measures Scale of Measurement Stevens 1946 Four types nominal ordinal interval and ratio Nominal scales set of unique cases types or categories with NO ORDER Only nonparametric operations are valid counting frequencies modes chisquare pointbiserial correlation Ordinal scalesm Features of Measures Scale of Measurement Stevens 1946 Interval 39ntervals of the scale are equal 39n magnilude gesc sessay but not stt cient oondition for parametIic statistica valid operations all mathematical operations means standad deviations em may be calculamd If other distrbutional asrmptions are net l39nea md nonrl39near regression ttests ANOVA ae also valid no fundamental Home ratio stamments allowed Ratio Lke 39nterval but also has a fundamental zero po39ntiallows ratio mmene srcn as A is twioe as much as Bquot Generally 39nterval or ratio scales should be used if possible More powerful and exible statistical tesE More precision 39n evaluatirg quantitative hypothaes rsvcm Research Methods Features of Measures Sensitivity Sensitivity measure must show changes in response to changes in the independent variable Range effects Ceiling effects variable reaches its highest possible value and gels truncated test is too easy Floor effects variable reaches ils lowest possible value and gels truncated test is too hard rsvcm Research Methods Features of Measures Re I ia b i ity the ability of a meailre to produce cons39stent talks M181 repeated measuemenE ae taken under identical conditions ypes precision physical meailrement 1noise niargh o error samplhg in surveys inmrrater reliability observers viewing the same behavior Testremst parallel forms and splithalt reliabilities psychological mse stc lz Wigwam Other Features of Measures Accuracy does a measure produce results that agree with a known standard Accuracy Vs Precision Validity Measurement Validity the extent to which your measure indeed measures what it is intended to measure Types Face Validity Content Validity Criterion related Validity concurrent Vs predictive Construct Validity Relationship between reliability and Validity sycsiz Resulth Methods Probability and Statistics Visualizing Variability Distributions of Frequency and the Histogram Why are probability and statistics important Histograms used to represent Bn Begum T egtto ais essvaiability in data frequencies of data in different D D re men arlance Variability die to different levels of 39ndependent Classes quot magmas l D variable 2 El Good variance that we Wmt to maximize E 3 D I Error Vaiance Variability 39n data due to facmrs othe thm the 3 4 3 treatment g b 5 i I Bad Vaimce that we want no m39nimize E B B Probability and Statis ts ae sinply tools used to am and l 2 7 4 oompae these sources of vaiability a 2 uizatsoiaaiu 9 B Grade in l rsycm summer rsycm summer Displaying Histograms Stem and Leaf Plots Distributions of Probability Density Stern and Leaf plots are used to display histograms graphically on their side using only typed characters Similar to frequency tem hypothetical histogram for IQ hlsmgram exoeli t quotaxls 5 now represents 3 7 35668 probability density 3 3 s 012234445555667777889 mass rather than 3 9 00011233333334445566667889999 frequency E II 10 01112233334444445566677777888899 g 11 01131122233444566777899 i l 12 Probability density H ll Fl 13 FrequencyN n i 2 3 6 5 a 7 a a in Grade stc lz Wigwam stc lz Wigwam F mhahililvDensW Some Types of Distributions Normal Gamma F mhahililvDensW stc iz Wigwam Measures of the Center of a Distribution Measures of center represent the general magnitude of scores in a distribution Mode most frequent score Median the middle soore of an ordered distribution Mean average where X is the data and N is the total number of observations stc iz Wigwam Measures of the Spread of a Distribution Measures of spread are used to assess the consistency of soores in a distribution Range max score m39n soore Interquartile range scoreQ3 scoreQl Variance a and standard deviation 7 and N is the total number of observations stc iz Wigwam More on Variance Standard Deviation Q sqrtvariance X is he data m is the mean of the data and N is the total number of observations Why N instead of Nl Populations vs Samples Remembering how to compute variance the mean of the squares square of the meansquot stc iz Wigwam Dacribing Distributions Parametrically Statistical Moments Any distribution based on interval or ratio data can be summarized by its statistical moments First Moment Mean location of distribution on xaxis Second Moment Variance dispersion of distribution Third Moment skewness symmetry of distribution Fourth Moment Kurtosis degree of peakedness stc iz Wigwam Estimators Sanple siatistics estimatepopulation paametevs Mem vs v56 Cy uses all 39nforma on n sample mean and vaiance ae simcient mode and range ae no Unbiasedness expected value approad1es real value wiin increased samping Efficiency ughina of clusmr of sample slapstics reIau39ve to the populau39on paame Resistmce inmence of ouuiers on sample siatisu39c stc iz Wigwam Next Time Topic descriptive statistics variables sampling and more on hypothesis testing Be sure to Read the assigned readings Howell chapters 34 Continue searching and reading the scientific literature for your proposal stcaiz ResuvchMelhads Lecture 15 Outline Review of Lecture 14 Multi factor research designs Today other specialized research designs Statistical nuts and boltsquot in conducting hypothesis tests of means C mbinin ex rimental and correlational UniZigg39ogigeaho designs Enalyegis of covariance or ANCOVA QuaSlExperlmental DeSIgns Developmental Desig s Smalln designs and Psychophysical Methods PSYC512 Research Methods stc iz summoning stc iz summoning Statistical nuts and boltsquot in Does a sample mean significantly testing hypotheses about means differ from a population mean The centIal limit theorem The mean ofthe sanpli mean of the population rtquo The variance of the samEaling distrbution of means is equal to the vaianoe of the popu tion from which the sanples were drawn divided by the size of the samples Exer e standard denation d39strbution of means is equal m the o of population is known m wlnidn the samples were drawn of sample mean o of population is estimated as s itth 39 39 latio 39 distr39b ted 39 it39 bell 39 at ofsample mam eonnaou nis iu norma ie is shaped e sgm pling distribution of means will also be 39 Qgp39yoggggagles 39quotm39tY39 t normal If the ori hal population is not normally distributed the sa 39ng distr ution of means will increas39ngly For small N ds fbuhon of approximate a normal d39stribution as sanple size 39ncreases same ame 5 Skewed ie winen 39ncreas39ngly lage samples ae drawn N Df stc iz summoning stc iz summoning The ttest Specialized Research Designs Combining betweensubjecls and withinsubjecls factors in Differences in means of matched samples resea39d des39gquot m39XEd 5399 Combining experimental and correlational designs Analysis of covariance or ANCOVA QuasiExperimental Des39gns Differences in means of Independent samples Pretest 5 st designs Developmental designs Longitudinal or crosssectional stc iz summoning stc iz summoning Combining Experimental and Correlational Designs Covariams in experimental designs Measire your sibjeas on a ciJvariate a vaiable that you believe may be correlated with your dependent variable aiams add error vaianoe and might Measir39ng the covariam allows you m use correla mdiniques in your malys39s e g malysis of Cova ANCOVA m ilbtract outquot the error varimce associated with the oovariam thereby 39ncreas39ng the statistical power of your experiment tional statistical rimce or Example measu39ng IQ 39n a lean39ng experiment rsyom Research Methods Combining Experimental and Correlational Designs Quasiindependent variable in experimental designs Quasi mea kind of but not reallyquot Similar to including a covariate exce ns pt merit of covariate is used to assign Ss to Covariate is thus treated as an quasiindependent variable Quasiindependent variables are referred to as quasi 39 ated they are measures that are treated as independent variables in the experimental design and analysis r because they cannot be manlpul essentially dependent variables mm Research Methods Quasiexperimental Designs Quasiexperimental designs are those in which only quasi independent variables are use Time series vs pretestposttest desig s Time series Measure behavior several times prior to and following a treatment time ser39 design or change in your quasiindependent variable interrupted time series design Pretestposttest Measure behavior once prio to once following the change in your independent variable rsyom Research Methods Quasi experimental Designs Equivalent time samples design Timeseries design especially useful for treatments with transient effect Repeatedly measure behavior following multiple applications and withdrawals of the treatment Nonequivalent control group des39gns helps control fo r history confounds which should affect both groups equally rsyom Research Methods Developmental Dsigns Used m mess d1mges 39n behavior relamd to a person s chronological 9 serves as a qias39rindepeideit vaiable Crosssectional desigm s m ltaneousiy mst sibiects Eigned m two or more age goups Generational effects cai confound the age variable Longitud39nal designs 5 m at est a s39ngle group of sibjeas over time Cottrols for generational effectsJout may still limit external validity May be confounded by hismry mortality andor multiple observation eff c Cohortssequential desi n Combines longitudinal and crosssec onal designs by meastring multip age groups over 39me 39 h a ws evauation of generational or hismrical oonfoun s 7mm Research Methods Single Subject Research Designs Research that focuses on identifying functional relationships between variables and performance of a single subject eg behavioral analysis and psychophysics Typically involve Large number of observations Rigid experimental control Investigations of powerful variables whose effects are easily detected rsyom Research Methods Baseline Designs Same as timeseries design Time series Measure behavior several times prior to and following a treatment Two phases A and B gm A baseline phase to establish behavioral baseline performance on DV prior to w treatment requires that a V stability criterion be reached W n B intervention phase that 2 4 mm measures performance on DV mam ResumeHypotheticalWWW cation Study Showing Intervals oFAttentive Behavior lorTwo a r Students During Baseline and Intervention Phases 3U Baseline Intervention ntervals of attentive be or 0 6 8 10 12 14 16 PSYC512 Research Methods Baseline Reversal Designs Baseline A intervention B l Baseline A Problem Time confound I Solution ABA Design reverse the treatment by removing it and see if performance returns to baseline unlikely to occur by coincidence s Intervention B l l I I I I l I l I I l I I I l I I I I I I l I I l I I l I I I I l I I l l l l l l I I I I l l l l l l l I I I I Problem with reversal I I 0 I ll l I I I I l I l I II I l l i II I l I l I i now IS 2 4 6 8 10 I2 14 16 13 20 22 24 26 28 30 Class ei iods FIGURE 11 5 Results of Hypothetical Behavin Modi cation Study Following an ABAB Design Solution ABAB design A B A B PSYC512 Research Methods Multiple Baseline Designs Used to assess irreversible changes in behavior Assess multiple independent behaviors and introduce treatment to only one behavior at a time Controls for time effects history maturation PSYC512 Research Methods Discrete Trials Designs Psychophysical Techniques Used to determine thresholds and difference thresholds justnoticeable differences or JNDs 55 receive dozens or hundreds of trials under tightly controlled conditions Methods Method of Constant Stimuli Method of Adjustment Method of Limits PSYC512 Research Methods Classical Psychophysical Methods Fechner s Elements of Psychophysics 1860 Absolute threshold limen how much energy must exist in a stimulus for it to be detectable Subliminal below threshold Superliminal above threshold PSYC512 Research Methods Methods for Determining Thresholds Method of adjustment Intensity or feature of stimulus is changed until it matches a standard Hysteresis requires both ascending and descending trials Average match across ascending and descending determines threshold Fast but least accurate PSYC512 Research Methods Methods for Determining Thresholds Methods for Determining Thresholds Method of Limits discrete method of t Method of Constant Stimuli adjustmen 39 Choose 59 stimuli Like method of adjustment except me ab VeI 5 me c below threshold 25 adjustment is done In discrete steps whose Present in random M Size is controlled by the experimenter der aquot 3 Threshold Hysteresis requires both ascending and threshold equalsl 39 39 Ll descending trials I energy that detected Ammuniarsmmsamgy Average match across ascending and 50 of the time descending determines threshold Slowest but most I For a Variant Staircase method II Methods sensitivity 1threshold stcaiz ResuichMeMads stcaiz ResuichMeMads Next Time Oncampus Live students please email me article presentation time slot preferences stcaiz ResuichMeMads PSYC512 Research Methods Brian P Dyre University ofIdaho rsysm Research Methods Lecture 14 Outline Review of Lecture 13 Multifactor research using two or more independent variables Mixed Designs More on research design Covariates and Quasiexperimentation Smalln designs and Psychophysical Methods rsysm Research Methods Combining Experimental and Correlational Designs Covariams in experimental designs Meamre your ilbjeds a covariate a vaiable that you believe may be correlated wiui your dependent variable fleit unmeaaired thae covaiams add error vaianoe and might obscure significmt effects Measir39ng the covariam allows you m use correlational statistical mdiniques in your malys39s e g malysis of covariaice or ANCOVA m ilbtrati out the error variaice associated with the covariam thereby 39ncreas39ng the statistical power of your experiment Example measu39ng IQ 39n a lean39ng experiment rsysm Research Methods Combining Experimental and Correlational Designs Quasiindependent variable in experimental designs Quasi means kind of but not reallyquot Similar to including a covariate except merit of covariate is used to assign Ss to Covariate is thus treated as an quasiindependent variable Quasiindependent variables are referred to as quasi because they cannot be manipulated they are essentially dependent variables measures that are treated as independent variables in the experimental design and analysis r y s 512 Research Methods Quasiexperimental Designs Quasiexperimental designs are those in which only quasi independent variables a Time series vs pretestposttest designs Time series Measure behavior several times prior to and following a treatment time series desi n r change in your quasiindependent variable interrupted time series design Pretestposttest Measure behavior once prio to and once following the change in your independent variable rsysm Research Methods Equivalent time samples desig Nonequivalent control group des39gns helps control for Quasi experimental Designs ri Timeseries design especially useful for treatmenls with transient effect Repeatedly measure behavior following multiple applications and withdrawals of the trea merit history confounds which should affect both groups equally rsysm Research Methods Developmental Designs Used to assess changes in behavior related to a person s chronological age which serves as a quasiindependent variable Crosssectional designs Simultaneously test subjects assigned to two or more age groups Generational effects can confound the age variable Longitudinal designs Repeatedly test a single group of subjects over time Controls for generational effects but may still limit external validity May be confounded by history mortality andor multiple observation effects Cohortsequential design Combines longitudinal and crosssectional designs by measuring multiple age groups over time which allows evaluation of generational or historical confounds PSYC512 Research Methods Single Subject Research Designs Research that focuses on identifying functional relationships between variables and performance of a single subject eg behavioral analysis and psychophysics Typically involve Large number of observations Rigid experimental control Investigations of powerful variables whose effects are easily detected PSYC512 Research Methods Two phases A and B Baseline Designs Same as timeseries design Time series Measure behavior several times prior to and following a treatment 30 4 Baseline Intervention h N o I Intervals of atten A baseline phase to establish i behavioral baseline vquot performance on DV prior to treatment requires that a stability criterion be reached OWW B Intervention phase that 2 4 aassgmdj 2 4 16 measures performance on DV after treatment FIGURE 114 Results of a Hypothetical Behavior Modi cation Study Showing Intervals oFAttentivc Behavior forTwo Students During Baseline and Intervention Phases PSYC512 Research Methods Baseline Reversal Designs Problem Time confound Solution ABA Design reverse the treatment by removing it and see if performance returns to baseline unlikely to occur by coincidence Problem with reversal now behavior is at baseline again Solution ABAB design Intervals ofaitentive behawor i I i 4 8 Design A PSYC512 Research Methods A i l l l l l I i ll i l i 10 12 14 i6 18 20 22 24 26 28 30 iii FIGURE 11 5 Results of Hypothetical Behavior Modi cation Study Following an ABAB B Multiple Baseline Designs Home samplej i 80 9mDntil Followup Used to assess irreversible changes in behavior Assess multiple independent behaviors and introduce treatment to only one behavior at a time Controls for time effects history maturation FIGURE 1 1 1 0 Percentage of Correct Consonant Production Within Spunr t n ous Speech Production forTIiree Consonants for Participant BH SOURCE Camaiaia 1993 reprinted with permission PSYC512 Research Methods Discrete Trials Designs Psychophysical Techniques Used to determine thresholds and difference thresholds justnoticeable differences or JNDs 55 receive dozens or hundreds of trials under tightly controlled conditions Methods Method of Constant Stimuli Method of Adjustment Method of Limits PSYC512 Research Methods Classical Psychophysical Methods Fechner39s Elemens of Psychophysics 1860 Absolute threshold limen39 how much energy must exist in a stimulus for it to be detectable SAblxmmal below threshold mpzrlxmmal abovz threshold stcmz Mammal Methods for Determining Thresholds Method of adjustment Intensity or feature of stimulus is changed until it matches a standard Hysteresis requires both ascending and descending trials Average match across ascending and descending determines threshold Fast but least accurate stcmz Wigwam Methods for Determining Thresholds Method of Limits discrete method of adjustment Like method of adjustment exce t adjustment is done in discrete steps whose size is controlled by the experimenter Hysteresis requires both ascending and descending tria s Average match across ascending and descending determines threshold Variant Staircase method stcmz Wigwam Methods for Determining Thresholds Method of Constant Stimuli Choose 59 stimuli some above some below thre n I I Present in random order 5 P I threshold equals amount 0 n th stimulus ener h t detected 50 of the time Slowest but most accurate I For all Methods sensitivity 1threshold Amanm ammiwg ngy stcmz Wigwam Next Time More on experimentation Smalln designs stcmz Wigwam

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "I used the money I made selling my notes & study guides to pay for spring break in Olympia, Washington...which was Sweet!"

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.