### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Applied Econometrics ECON 753

UMass

GPA 3.74

### View Full Document

## 18

## 0

## Popular in Course

## Popular in Economcs

This 41 page Class Notes was uploaded by Mr. Kay Bergstrom on Friday October 30, 2015. The Class Notes belongs to ECON 753 at University of Massachusetts taught by Michael Ash in Fall. Since its upload, it has received 18 views. For similar materials see /class/232316/econ-753-university-of-massachusetts in Economcs at University of Massachusetts.

## Similar to ECON 753 at UMass

## Reviews for Applied Econometrics

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 10/30/15

Applied Econometrics Critical Replication Michael Ash Econ 753 Critical Replication 7 p14 What is critical replication Subject economic and econometric results to tests for accuracy robustness and validity 1 Replicate the original result then 2 Extend By applying the same model to new data different years different country different countries different or placebo policy different frequency aggregation source or quality of data or 0 By applying a different model to the same data eg political economy raceclassgender democracy environment functional form nonlinearity categorization definitions outliers estimation method Critical Replication 7 p24 More on Critical Replication 39 Why replicate 0 Test for simple errors 0 Understand the procedure 39 Why extend 0 Test internal or external validity 0 Test the causal model 0 Reassess with a new paradigm 0 Some examples in Econ 753 0 Update of FeldsteinHorioka savingsinvestment model 0 Reassessment of LevineZervos Critical Replication 7 p34 Data for Critical Replication Alternative strategies 1 Assemble your own from the author s published instructions hard Use raw data and author s programs 3 Acquire final dataset from author Critical Replication 7 p44 Assessing Studies Based on Multiple Regression Stock and Watson Chapter 7 39 Internal Validity statistical inferences about causal effects are valid for the population being studied 0 What would happen to California elementary school test scores if every district reduced the STR by two students 0 External Validity statistical inferences about causal effects can be generalized from the population and setting being studied to other populations and settings 0 What would happen to California HS test scores if every district reduced the STR by two students 0 What would happen to Iowa elementary school test scores if every district reduced the STR by two students 0 What would happen to Japanese elementary school test scores if every district reduced the STR by two students Critical Replication 7 p54 RodRod Trade Policy and Economic Growth A Skeptic s Guide to the CrossNational Evidence Francisco Rodriguez and Dani Rodrik NBER Working Paper 7081 1999 and NBER Macroeconomics Annual 2000 39 Do countries with lower policyinduced barriers to international trade grow faster once other relevant country characteristics are controlled for Large empirical literature providing an affirmative answer 0 Dollar 1992 BenDavid 1993 Sachs and Warner 1995 Edwards 1998 Critical Replication 7 p64 RodRod Begins with a small learningbydoing model of ambiguous effect of trade policy Prima facie case is weak Figs L1 and I2 Definition of trade barrier Focus on trade policies rather than volume of trade Data quality Relevant outcome GDP v welfare Publication bias Critical Replication 7 p74 Figure 11 Partial Association between Growth and Import Duties 05203 KOR THA BWA Growth unexplained part 7 NIC O 01 J 1 01 M i i 018885 514781 import duties as of imports Weak relationship between tariffs and growth Critical Replication 7 p84 Figure 12 Partial Association between Growth and NonTariff Barriers 049784 7 THA KOR OAN IDN m CYP MYS EGY PAKHL PRT IND IRL NPL BGD TUN pRY JFILKA ESP COL TUR 3N GRC SYR Aeggg NK BE ECU CRI Dali R BEL CHE NARG PHL GTMZWE s New W BR J GHA Tro MWI ELEM NY TZA PER HTI IRN MB RWA NIC 055044 J O 888 nontariff barrier coverage ratio Weak relationship between NTB s and growth Critical Replication 7 p94 RodRod 39 Example Dollar 1992 0 DISTORTION index of real exchange rate distortion based on LOP 0 VARIABILITY index of real exchange rate variability 0 Do not necessarily measure trade policy Role of ExchangeRate Policy 0 Devaluation strategy incorrectly associated with low distortion 0 Does not include initial income education regional dummies standard variables in crosssectional growth literature 0 Not robust to data revisions PWT 56 instead of 40 O Outlier analysis Distortion effect driven by Ghana and Uganda Critical Replication 7 p 104 RodRod Findings and Recommendations 0 The relationship between trade barriers and economic growth receives weak support from the evidence Results are contingent on dubious definitions specific data outliers 0 Explore contingent relationships different effects of trade policy in low versus highincome countries mfg versus primary economies dependent on world economy 0 Disaggregate policy and explore more carefully tariff capital controls exportprocessing zones etc 39 Plantlevel datasets especially does export cause efficiency or efficiency cause export Critical Replication 7 p1 14 Internal Validity 39 Want B to be unbiased and consistent estimator of B 0 Hypothesis tests should have the intended significance level and Cl s should have the desired confidence level 0 Depends on SE s being accurately estimated 0 Example of a threat to internal validity omitted variable bias 0 Solution include omitted variables Critical Replication 7 p 124 External Validity Example 0 Laboratory animal toxicity studies to study predict and regulate human exposure and health effects What can go wrong Differences in Populations between population studied and the population of interest geography time eg RAND HIE 0 Differences in Settings legal institutional and physical environments Test scores and STR 39 ES scores in the US more likely to be an externally valid application than HS scores in US or E8 scores in Japan 0 Except Highstakes testing for students teachers etc Critical Replication 7 p 134 Assessing External Validity Requires 0 Specific knowledge of the population and setting studied and the population and setting of interest or 0 Studies on several populations and settings that generate similar results Critical Replication 7 p 144 Threats to Internal Validity and Solutions 39 Omitted Variable Bias 39 Misspecification of the Functional Form 0 Imprecise Measurement of the Independent Variables ErrorsinVariables Sample Selection 39 Simultaneous Causality Each of these is an instance of correlation between the regressor 0 and the error term u which violates the first least squares assumption Critical Replication 7 p 154 Omitted Variable Bias Already discussed the problem at length 0 Solution when Omitted Variable is Observed 0 Identify the key coefficients of interest 0 Consider which control variables to include based on expert judgment 0 Estimate alternative specifications keep additional variables that 0 are themselves statistically significant or affect the sign size or significance of the coefficients on key variables 0 Full disclosure of the specifications tested 0 Solution when Omitted Variable is Unobserved 0 Compare a unit to itself over time or within superunit 0 Experimental or quasiexperimental design Critical Replication 7 p 164 Misspeci cation of the Functional Form Curved and changing relationships Chapter 6 beyond the scope of this course Use scatterplots to identify nonlinear relationships 39 Discrete outcome variables Chapter 9 Critical Replication 7 p 1 74 Errorsin Variables 39 Only a problem with imprecise measurement of the independent variables The imprecise measurement is not biased up or down simplyX is true signal but w is noise imprecise measurement ofX is added X Xw Leads to Attenuation Bias A B1 is always an underestimate of B1 estimated effect is closer to zero smaller than true effect 0 lmprecise measurement of the dependent variable is not a problem imprecise measurement is simply one of the other factors u that affect Y Critical Replication 7 p 1 84 Errorsin Variables cont d Solutions 0 Multiple independent measures ofX even if all are imprecise 0 Adjust estimates for attenuation bias based on estimated size of the imprecision Critical Replication 7 p 194 Sample Selection Availability of the data is influenced by a selection process that is related to the value of the dependent variable In all these cases other factors u may be correlated with X 0 1936 Presidential poll limited to car and telephone owners 0 People who apply forjobtraining programs likely have barriers to employment 0 People with jobs may have high earning potential controlling for their characteristics 0 lnnerChange program evaluation attrition in general Solutions various and complex create an explicit model of the selection process Critical Replication 7 p204 Simultaneous or Reverse Causality 39 Government may hire additional teachers in lowperforming districts or now government may penalize lowperforming districts Y BoBiXiui E WWKW Induces correlation between u and X 0 Consider case where ui is low hence Y is low 0 lfY is low then assuming 71 positiveX is low 0 But this means that ui and X are low together correlated Solutions randomized controlled experiments Chapter 11 and econometric quasiexperimental methods beyond the scope of this course Critical Replication 7 p21 4 Summary Note that every problem discussed so far involved a violation of OLS assumption 1 the conditional distribution of ui given X has mean zero 39 General language for discussing problems with causal models within and beyond econometrics Critical Replication 7 p224 Inconsistency in OLS Standard Errors 39 The OLS estimates ofB remain consistent and unbiased but 0 Inference Cl s hypothesis tests will be wrong because the SE s are wrong 0 Heteroskedasticity use robust standard errors 0 Correlation of the error term across observations Y BoBiXiui Y B0B1Xjuf ui and u should not be related 0 Repeated sampling of the same unit over time serial correlation 0 Sampling within the same household or geographical unit 0 Less fewer observations than meets the eye Critical Replication 7 p234 Internal and External Validity 39 External Validity 0 Internal Validity O O O O Omitted Variable Bias Misspecified Functional Form ErrorsinVariables Sample Selection InnerChange example Simultaneous Reverse Causality Inconsistent Standard Errors Critical Replication 7 p244 Sample Selection Availability of the data is influenced by a selection process that is related to the value of the dependent variable In all cases other factors u may be correlated with X 0 1936 Presidential poll limited to car and telephone owners 0 People who apply forjobtraining programs likely have barriers to employment 0 People with jobs may have high earning potential controlling for their characteristics 0 lnnerChange program evaluation attrition in general Solutions various and complex create an explicit model of the selection Critical Replication 7 p254 InnerChange Goal evaluation of nnerChange IFI a religionbased rehabilitation program for prisoners Method compare outcomes for nnerChange treatment group and several comparison groups Nonexperimental assignment to treatment Outcome variables 1 percent rearrested within two years 2 percent incarcerated after two years H0 1 PIFI PControI H1 plFl pControl Critical Replication 7 p264 InnerChange Statistics gt Summaries tables amp tests gt Classical tests of hypotheses gt Twosample proportion calculator Two sample test of proportion x Number of obs 177 y Number of obs 1754 Variable I Mean Std Err z Pgtz 95 Conf Interval x 243 0322377 1798152 3061848 y I 203 0096042 1841761 2218239 diff 04 033638 0259292 1059292 under Ho 031934 125 0210 Ho proportionx proportiony diff 0 Ha diff lt 0 Ha diff l 0 Ha diff gt 0 z 1253 2 1253 2 1253 P lt z 08948 P gt z 02104 P gt z 01052 Critical Replication 7 p274 InnerChange Sample Selection Bias Analysis above is based on IntentionToTreat The program evaluation of InnerChange also reported 0 The average outcomes of F graduates who completed the 16month program in prison and after release and found employment were better than the average outcomes of the comparison groups 0 Consider the outcomes of F graduates as TreatmentOnTreated TOT 0 Conditioning the outcome on graduation is selecting the sample on factors that are highly correlated with the outcome variable 39 In this case the selection of F graduates selects winners among the treatment group Critical Replication 7 p284 InnerChange Sample Selection Bias 39 Why isn t TOT interesting in this case 0 Because the untreated within the treatment group IFI participants who did not graduate did substantially worse than the control group Two possibilities 1 F actually harmed some participants while helping others 2 The program had no effect but selecting graduates selected for good outcomes Critical Replication 7 p294 Simultaneous or Reverse Causality 39 Government may hire additional teachers in lowperforming districts or now government may penalize lowperforming districts Y BoBiXiui E WWKW Induces correlation between u and X 0 Consider case where ui is low hence Y is low 0 lfY is low then assuming 71 positiveX is low 0 But this means that ui and X are low together correlated Solutions randomized controlled experiments Chapter 11 and econometric quasiexperimental methods beyond the scope of this course Critical Replication 7 p304 Summary Note that every problem discussed so far involved a violation of OLS assumption 1 the conditional distribution of ui given X has mean zero 39 General language for discussing problems with causal models within and beyond econometrics Critical Replication 7 p314 Inconsistency in OLS Standard Errors 39 The OLS esimates of B remain consistent and unbiased but 0 Inference Cl s hypothesis tests will be wrong because the SE s are wrong 0 Heteroskedasticity use robust standard errors 0 Correlation of the error term across observations Y BoBiXiui Y B0B1Xjuf ui and u should not be related 0 Repeated sampling of the same unit over time serial correlation 0 Sampling within the same household or geographical unit 0 Less fewer observations than meets the eye Critical Replication 7 p324 Test Scores and STR External Validity Massachusetts and California 0 Although distinct both tests are broad measures of student knowledge and academic skill 39 Differences in elementary school funding and curriculum Districtlevel test scores on different tests 39 Higher average STR in California 196 in CA vs 173 in MA 39 Higher average district income in MA 0 Wider spread in income in CA 39 More poor students and English learners in CA Critical Replication 7 p334 External Validity Basic Characteristics Summary Statistics for California and Massachusetts Tesf Scare Dala Sels Average dkmcr Income 3 Number ofobscrvau39ons 420 199 22 199 0 8 IABLE 71 California Massachusetts Average Standard Daviaiion Average Standard Davialian Tex scores 6541 191 709 I54 Student eacher ratio 196 1 3 3 English learners 158 183 11 290 Receivng lunch subsidy 447 153 151 815317 18747 335808 Copyright 2003 by Pearson Education inc Cn39tical Replication 7 p344 External Validity MA Regression English lenrucls 4437 emu 1 3 H300 mm 71 p i I J Dam from Massachusens nependem M u I v 39 39 Fuunh Grade 220 Observaliuns Rogmssnr a A 5 a SmdLnHLuclm mno ANNquot 124 402 A7 S39I39R 027 140 137 027 5178 4mm 0737 37133 0011 H013 x En vi h Immm gt 42 median Binary 7 93 HrELx STR 180 msn Eligible for m Innrli n5 0582 4687 4mm uu77 n m7 0 um um I nimm Ilcolnc logimlun I053 1 15 Disnm KIICOIHC 338 387 249 249 Dntnct mayan 0th H74 H184 ms 1185 mm mm miss Dnuiu xllcumc ALUUZZ 410123quot runnzr my mum mum mum mum lntciccp 7396M 74m 6615quot 7599 747 36 213 x I 3 232 203 Table 72 wnlinued Copyright 2003 by Pearson Education Inc Critical Replication 7 p354 External Validity First case 39 California 0 Adding variables that control of student background characteristics reduced the coefficient from 228 to O73 a reduction of 68 percent 0 The null hypothesis H0 BSTR O was rejected at the 1 percent significance level Massachusetts 0 Adding variables that control of student background characteristics reduced the coefficient from 172 to O69 a reduction of 60 percent 0 The null hypothesis H0 BSTR O was rejected only at the 5 percent significance level but there are more observations in CA which tends to drive down the SE Critical Replication 7 p364 External Validity Truly comparable 39 Different tests different scores different differences in scores 0 One test point does not mean the same thing in MA and CA 0 Standardize the results by dividing the change in test score by the standard deviation of test scores 0 Expresses the effect on test scores in terms of the observed overall spread in test scores Critical Replication 7 p374 External Validity Summary Table and Massachusetts TABLE 73 StudenHeucher Ratios and Test Scores Computing the Estimates from California 0L5 Estimate Standard Davintiun of Test Scores Estimated Effect of 2 Fewer Students Per Teacher In Units of 5 Acmss Distrizts Points on the rest Deviations California Linear Table 22 73 19 0070 026 0027 Cubic Table 027 191 0153 Raina STRfrnm 20 m 18 0037 Cubic Table 627 7 91 190 0099 Redua39 STR mn 22 m 20 069 0036 Massachusetts Linear Tablc 723 064 15 128 0085 027 054 0036 Standard errors arc given in parentheses Copyright 2003 by Pearson Education inc 744 Cn39tical Replication 7 p384 Test Scores and STR Internal Validity Omitted variable bias we ve done our best English learners socioeconomic background Other important possibilities teacher quality extracurriculars family commitment to learning Next stop experiment Functional form Chapter 6 not this course Basic result is that linear is ok ErrorsinVariables student moves may lead to mismeasurement of district STR income data are from 1990 Census while other data pertain to late 1990 s Selection all districts included Absenteeism Simultaneous causality for example allocation of resources based on test performance either compensatory or sanctioning Neither California nor Massachusetts had late 1990 s significant allocation based on performance Critical Replication 7 p394 Test Scores and STR Internal Validity cont d 39 Heteroskedasticityconsistent standard errors 0 Universe of observations is not the same as a random sample the ideal under OLS Assumption 2 but probably ok Critical Replication 7 p404 Test Scores and STR Finally done Effect of STR cutting STR by two students increases test scores by approximately 008 standard deviations 0 Effect is signficant but small Result appears generalizable in US elementary school systems Consider other omitted variables listed above Basis for policy decisions estimating benefits in a costbenefit analysis of changing STR versus other possible uses of education resources Did we measure the right outcome Alternative outcomes Critical Replication 7 p414

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "Knowing I can count on the Elite Notetaker in my class allows me to focus on what the professor is saying instead of just scribbling notes the whole time and falling behind."

#### "When you're taking detailed notes and trying to help everyone else out in the class, it really helps you learn and understand the material...plus I made $280 on my first study guide!"

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "It's a great way for students to improve their educational experience and it seemed like a product that everybody wants, so all the people participating are winning."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.