TVR 263 Mass Media Research Methods Exam #1
* # - based on the review sheet
Mass Media Research Phases/Characteristics of Sciences, etc.
August 29th 2016 reading/presentation
Define Mass Media
● Old definition: Any communication channel used to simultaneously reach a large number of people, including radio, TV, newspapers, magazines, billboards, films, recordings, books and the Internet.
● New definition: same as the above ^, add smart media
● An attempt to discover something → the way to find out what people want. Mass media research:
- How do viewers evaluate a pilot for a new TV show?
- Are there more violent acts on TV now than five years ago?
- What kinds of people watch TV online?
*decisions in the media industry are based on research and data*
● “Ways of collecting data” eg. what you actually do in order to collect data and carry out an investigation.
‘Cool’ Methods: eye tracking, virtual reality.
*1 - Mass Media Research Phases
● Phase 1 - the medium - Interest in the medium itself. What is it? How does it work? What technology does it involve?
● Phase 2 - medium uses - Research begins once the medium is developed. How do people use the medium in real life? Information? Entertainment?
● Phase 3 - medium effects - Investigation of the social, psychological, and physical effects of the medium. Does it change people’s perspectives about anything? ● Phase 4 - medium improvement - Research conducted to determine how the medium can be improved.
Research Phases example - page 6 - aug.29th reading.
*2 -The Methods/Sources of Knowledge/Knowing - pg.9
● Tenacity - you think something is true because it’s always been true for you. Not a good way of knowing something because time can change. E.g owning a store inherited from your family → grandparents and parents never had to advertise the store so why should they.
● Intuition - self evident to you eg. marketing manager who thinks they know what customers want. This is not good because the data collected could say something else. ● Authority - it’s true because everything the person in authority says it's true.
● Self - discovery - closest to scientific methods - when you learn for yourself what is true. What you learn by yourself is going to be private to you.
● Science - The Scientific method involve learning that comes with a series of small steps. Science has certain amount of characteristics.
*2 -Characteristics of Science Method Don't forget about the age old question of What was happening in europe in the 16th century?
Don't forget about the age old question of What is the meaning of trustworthiness in the speaker?
6 basic characteristics/ tenets distinguish the scientific method from other methods of knowing. (A research approach that does not follow each of these tenets is not a scientific approach) 1. Scientific research is public - free available information, available to other researchers on request.
2. Science is objective - Science tries to rule out eccentricities of judgement by researchers → scientific research deal with facts rather than interpretations of facts 3. Science is empirical - researchers are concerned with a world that is knowable and potentially measurable.
4. Science is systematic and cumulative - no single research study stands alone, nor does it rise or fall by itself. We also discuss several other topics like Who wrote the age of reason according to encyclopedia?
5. Science is predictive - science is concerned with relating the present to the future. 6. Science is self - correcting - series of small steps.
1. Select a problem
2. Review existing research and theory
3. Develop hypotheses or research Q’s
4. Determine an appropriate methodology
5. Collect relevant data
6. Analyze and interpret the results
7. Present the results in an appropriate form
8. Replicate the study (when necessary)
Research question: does television lead to distortions of reality for children? Hypotheses: a child’s level of distortion of reality is directly related to the amount and types of television programs the child views.
Hypotheses, Research questions, and variables
Aug.31 - reading/powerpoint
● Theoretical significance - relating/having the character of theory of a research study ● Societal significance - importance for society Don't forget about the age old question of How does artificial selection affect the environment?
● Personal significance - important to you
*3 What is a variable?
Simply put: variables are things that can change (or vary)
Can have more than one value along a continuum
- E.g., “satisfaction with pay per view programs”
Nb - variables might vary b/n people, location, time
*3 Hypotheses and Variables
- Most hypotheses can be expressed in terms of two variables
- Proposed cause and proposed outcome
Eg. Cold weather puts people in a bad mood
Warm weather puts people in a good mood
*3 Types of Variables
Proposed cause = independent variable
- Does not depend on other variables
- I.e weather, time of day, green/yellow skin
Proposed outcome = dependent variable
- Depends on the proposed cause
- I.e, mood
Step 1 - develop a research question
Step 2 - Identify variables
Step 3 - form your hypothesis/es
1. Will whatsapp user ratings increase of an introduction of ads?
2. Variables - independent - ad , dependent - users/ratings
3. Hypothesis - an introduction to ads will have no impact on users/ratings.
*3 Types of Variables pt.2
Another way to label variables
Independent variable = predictor variable
Dependent variable = outcome variable
Nb → independent variables causes a change in dependent variable. It is not possible the other way around.
Eg. time spent studying causes a change in test score. Test score can’t change the time spent studying.
*4 Operational definitions: define/explain how you plan to measure or test the variable *4 Levels of Measurement:
● Categorical variables
■ Nominal: categorical
■ Ordinal: items are ranked (still categorical)
● Continuous variables
■ Interval: numbers have equal distances
■ Ratio: numbers have proportional distances
*4 Types of Variables/research questions
Nominal variables - numbers used to classify things into categories . eg. gendder, sex, ethnicity, employment status,eye color, major
Ordinal variables -ranked . first, second, third, fourth eg. socioeconomic status → lower, middle. Upper.
Interval variables - equal disance b/n numbers but no zero point. Most research scales eg. this class is awesome 1. Disagree , 2. neutral, 3.agree.
Ratio variables - + true zero point - how many friends do you have? How old are you? Operational definitions and levels of measurement
Scale types; Rating, Likert, and Semantic Differential
Rating - rate a list of items, common in mass media research, type of scale 1-3, 1-5, 1-10
Likert - most commonly used , also called the summated rating approach. Statements are developed with respect to a topic, and respondents can strongly agree, agree, be neutral, disagree, or strongly disagree with the statement.
Semantic differential scales - aka bipolar rating scale - a lso commonly used, semantic differential technique. A name or a concept is placed at the top of a series of a seven point scale anchored by bipolar attitudes.
Eg. rating - rate a list of items , likert - I am __ in this class, semantic - how would you describe kmart on the following scale.
*6 Reliability vs. Validity
Reliability - results are consistent
Validity - results satisfy objectives
● Scales should be predictable, stable, and consistent
● Scales should give you a true measure and not error
● Two elements: stability and internal consistency
● Your instrument measures what it is designed to measure
● Face validity: “ah makes sense”
● Concurrent validity: how well your test compares to other objective measures
SPSS Statistics Reliability Analysis (in class activity)
● Stats anxiety questionnaire dataset
● Familiarize yourself with date & variables
● Check stats anxiety scale reliability
- Change the variable name
- Calculate the mean score
- Create a bar chart of responses
- Calculate the reliability of subscale
Probability vs. nonprobability & qualified vs. unqualified sampling
What are samples?
● Subset of the population that is a representative of the entire population → subset or small group that has the characteristics of the entire population. we want to make sure it has similar characteristics. NB. When we do get everyone from the population is a sensus.
● Probability vs. nonprobability
*8 Probability Sampling
● Uses mathematical guidelines for selection
● Representative of the population
Reduces sampling bias, results can be generalized
● Time-consuming and costly
4 different kinds of sampling:
1. simple random sampling - go and pick out people at random(entirely by chance) - everyone in this population have an equal chance of being in the sample.
2. systematic random sampling - line up and give everyone a number. start at person #2 and go to every 3rd person - somewhat random because we didn’t determine an order that their going in the line. (system but randomness)
3. stratified sampling - the population is divided into strata/groups. divided by the red,blue, and yellow people. Stratify sampling - preserving proportions that is in the population. 4. cluster sampling - divided into clusters/groups(countries,cities, etc) - then survey everyone in the cluster.
Does not follow the guidelines of mathematical probability
Convenience sampling - what we will do because it's what is convenient to access and collect data. - the characteristics of your sample would be homogenous (same kind/alike) E.g online/facebook surveys
Purposive sampling - characteristics of a population and the objective of the study. It is also known as judgmental, selective, or subjective sampling.
snowball/referral sampling - referrals to other who would be interested in taking part in your sampling (ask current participants)
Introduces sampling bias/error
● aka - self selected participants → can’t force people to take a survey or study you can only invite people.
● *8 Qualified -
control who is in the sample - eg. can send everyone a post evaluation from this course to just people who took your class.
● *8 Unqualified - ratemyprofessor - no control on who is participating this is important - if you are looking for a specific demographic eg. college students who uses instagram
● Probability sampling: ideal but not always realistic, possible or necessary - is time consuming - in this class it would be unrealistic
● Understand and be aware of your sampling method
● Control and know who is in your sample
● Report your sample clearly so the reader can interpret your results.
Ethical theories and principles
The importance of ethics really began around 1970s.
*9 Ethical Theory
● rule-based/deontological theories - rule based
● balancing/teleological theories - balance the good vs. the bad
● Relativistic theories - direct opposite of the first two stated, there are no ground rules - no universal standards - what’s ethical depend on the culture/standards of your area/discipline. Go by the cultural norms that are present.
Principles of Ethical Research
1. provide free choice to participants - you never should involve participants without active consent and not letting them know that they are participating in a research study. You always have to get consent and you can’t force people.
2. protect right to privacy - keep the data private don’t want it to be shared outside of your research team (raw data) - especially important to keep personal information private. 3. Benefit not harm - benefit participants dont hurt them
4. Treat with respect
● Participation should be voluntary and consensual
● Use deception cautiously and debrief afterwards
● Be clear about privacy and confidentiality
● Never manipulate date
● Informed consent - warn participants of risks/discomfort
● Absolutely can’t coerce participation
● New media and consent → is facebook a ‘public’ space?
Nb: facebook - changed their way of research in order for it to be more ethical . since it is a public space life cafe and train stations you don’t need consent from the people being observed but the situation gets difficult when looking at new media . for example having public posts that anyone can have access to.
- Anonymous vs. Confidential
- Anonymous - can’t link responses to a particular person
- Confidential - identity of participant is known but kept private
- This is less important for observations of public behavior/records
IRB → Institutional Review Board → responsible of reviewing all research and teaching activities conducted by IC faculty.
Summary: Importance of Ethics.
- You will have better data
- You will not have unhappy, alienated, or emotionally damaged participants
Survey Design: Close-ended vs. open - ended questions, writing questions, and survey administration
Surveys is best for information about people themselves, what they think and feel, do, etc. descriptive data- what the person is doing and feeling. However, it gets complicated when it’s time to find out the reason behind their doings.
The Survey Method
Interviews: face to face/telephone → dont tend to have a great response rate. Self- administered - online, paper surveys - you are administering the survey to yourself
● Inexpensive - requires little or no money/effort - able to distribute it to alot of people ● And surveys is all about numbers.
● Obtain current information
● Large amount of info
● Quantitative/numeric data
● Used often
● problem of self report - main disadvantage, people dont tell the truth ● low response rates - esp. for mail, telephone, and email
● Bad questions = bias answers - writing the questions can influence your answers - for
example: would you support killing innocent babies? vs. do you support women reproductive rights?
● bias answers - attract one specific audience can potentially affect it
*10 Constructing Questions:
Open-ended - force to explain more
Close - ended - can answer just yes or no
Tips to create questions
Enter the mind of who will take the survey - would it be confusing? Would they lose focus? Know how you plan to use/analyze answers - do i really need this question? What will this date determine?
*10 Bad Survey Questions
● Double - barreled question - bad because it ask 2 questions at the same time or ask more than one topic at once
● Leading question - leads them to answer the question a certain way ● Loaded question - fishing and directing you to answer the question a certain way ● Overly complex language
● Participants dont know the answers
● Answers not mutually exclusive
● Answers not exhaustive - exclude the ‘other’ option
● Overly personal or threatening question
● Off topic question
1. Start with an easy question
2. Important variables in the beginning
3. One topic at a time
4. Transition b/n topics
5. Demographics questions in the end
*10 Start with:Administering Surveys
Length (20-30 minutes max)
Pre-testing (Pilot survey)
- Do people understand the questions?
- Are you getting the info you want?
- Should you include more qs?
- Should you take out some qs?
- Does the survey method work?
*11 Content analysis
“It analyzes the content of something”
Content Analysis Method
● Systematic procedure used to examine content of recorded messages ● Study of social artifacts (human creations;e.g., books,laws,art,media) analyzing content in a systematic way. recording messages can be newspaper articles, television programs, books, magazines, art, etc. Anything with recorded content can be analyzed.
Content analysis: is a technique for systematically describing written, spoken or visual communication. It provides a quantitative (numerical) description. Many content analyses involve media - print (newspapers, magazines), television, video, movies, the Internet.
Uses of content analysis
Describe communication content (classify content into categories)
Test hypotheses about message characteristics
E.g comparing media content to the “real world”
Establish starting point for studies of media effects
● unobtrusive - dont have to ask people to collect data , the data is already out there. example: local library for old newspaper records.
● Relatively inexpensive - data may not be expensive but may be time consuming and takes more effort.
● current events/ present-day topics: f or example - emmy’s awards and get load of information
● Data is easy to obtain
● Quantifiable data
Must be reinforced by other studies
Hard to obtain reliability in coding
Hard to analyze things with no records (little media coverage)
Nb. everything but the data collection process will be time consuming
Steps of Content Analysis
1. Formulate research Q or hypothesis
2. Define the universe in question
3. Select an appropriate sample
4. Select and define a unit of analysis
5. Construct categories of content to be analyzed
6. Establish coding system
7. Train coders and conduct pilot study
8. Code content
9. Analyze coded data
10. Draw conclusions
Constructing Coding Categories
• Mutually exclusive: one theme per category (should not overlap)
• Exhaustive: all themes need to be covered
• Reliable: all coders agree
My notes: 2 people to code the same content but they are not getting the same results - reliable: all coders agree. it is important to make sure it doesn’t happen. make categories specific with a lot of detail and to train them alot. have them go through examples.
Detailed coding system helps
Independent coding → discussion
90% mutual agreement
Coders work on a small # of units first (pilot study)
New Media and Content Analysis
• New media has a LOT of content that can be analyzed
– e.g., Facebook posts, tweets, blog posts, online news articles, etc.
• Automated content analysis and coding using computer programs
• New frontier – content analysis of images (especially when it is automated)
My notes based on slide:automated content analysis that doesn't involve human coding is becoming more popular especially when it involves alot of data. for example a human vs a computer reading a thousand tweets. the computer program can target certain words. however, the computer can pick up nuance and meaning that only people can do. ex.”were so happy that donald trump was running for president” but it’s sarcasm. the computer would view it as a positive comment even if it wasn't. So the computer doesn't do well as people in terms of figuring out the meaning behind the data.
← content analysis of skype convo
*12 Experiments: Advantages/Disadvantages; Understand independent (manipulated) vs. dependent variable (optional reading available on sakai)
Advantages and Disadvantages of Lab Experiments.
Establish cause and effect
Control environment - you as the researcher can control the location, the participants, and what will happen once they get there.
Artificial nature of experimental environment - creating a situation that wouldn’t actually happen in real life.
Researcher (experimenter) bias
Limited scope - limiting the kind of questions you can answer
Steps to conduct experiments
1. Select the setting
2. Select the experimental design
3. Operationalize variables
4. Decide how to manipulate independent variable
5. Select and assign subjects to experimental conditions
6. Conduct a pilot study
7. Administer experiment
● The ‘spy’ in an experiment. Eg. actor got shocked by particpant in miligram’s obedience
experiment (also can be used as an ethics example)
● Person working with the research team.
Premises of the study
● “Pregiving message”: providing an unsolicited favor before making a request ○ Makes target more likely to accept request
● How does this work?
● Would this also work in a romantic context? (i.e., asking people out on a date) ● How does socioeconomic status play a role?
Design an experiment
● Variables - independent, dependent
● Create a hypotheses