Theoretical Statistics STAT 210A
Popular in Course
Popular in Statistics
This 6 page Class Notes was uploaded by Floy Kub on Thursday October 22, 2015. The Class Notes belongs to STAT 210A at University of California - Berkeley taught by Staff in Fall. Since its upload, it has received 69 views. For similar materials see /class/226735/stat-210a-university-of-california-berkeley in Statistics at University of California - Berkeley.
Reviews for Theoretical Statistics
Report this Material
What is Karma?
Karma is the currency of StudySoup.
Date Created: 10/22/15
STAT 210A Theoretical Statistics Fall 2006 Lecture 17 7 October 26 Lecturer Martin Wainwright Scribe Kurt Miller Warning These scribe notes have only been mildly proofread Outline 0 Con dence intervals 0 Uses of asymptotics asymptotic con dence intervals variance stabilization 1 step ef ciency 1 71 Con dence intervals Motivation Thus far we have talked mainly about point estimators 6 X a 9 of a parameter 0 It is often useful to have uncertainty information associated with an estimator One method is via con dence intervals 9 E R or sets in higher dimensions De nition Given 04 E 017 say that a random set SX is a 1 7 a con dence set if we 6 SX 217 a W e 9 171 In words 6 is in the random set SX with probability 2 1 7 a If 171 holds with equality V0 6 9 then we have an eacact con dence set In certain typically classical set tings7 we can obtain exact con dence intervals in closed form Example 1 X1an391n iid Nn1 We know that ZEXn7t N01 Z is called a pivot You cant actually compute it since it depends on the unknown it but its useful since its distribution is independent of M Given 04 Choose 2012 such that M2 2 2042 1 Zea2 042 17 1 STAT 210A Lecture 17 7 October 26 Fall 2006 where is the standard normal CDF Hence MWan 7m S 2012 17047 which implies is an exact 1 7 a con dence interval for it Note Regarding the interpretation of a frequentist con dence interval it does not mean that for a given realization X the actual parameter falls in the above interval 1 7 00 of the time Rather7 it means that if this experiment were to be run over and over again generating one con dence interval for each repetition7 then 1 7 00 of the intervals would contain the correct parameter Example 2 Suppose that X17 7X 239 171 iid NM7027 a location scale model where both it and 02 are unknown Again Xn Z 7N01 WE ltgt 71715 V 2 X371 039 where S Z1Xi 7 X These are both pivots that require knowledge of the parameters Claim Z and V are independent by Basu7s Theorem Look at T ZxVn 7 1 which yields a student t distribution with d n 7 1 degrees of freedom with T N td where d P 1 2 V3 1 3 We can use this to set levels Note T Xn 7 so if tutWEI is an 042 upper quantile7 then Pi Sn 7 Sn Xn 7 Eta2m7l7 Xn i ta2m71 is exact 1 7 a con dence interval for u for the normal location scale model fTW M ltta2n71 1a Six i Hence 172 Asymptotic con dence sets Motivation STAT 210A Lecture 17 7 October 26 Fall 2006 ln general7 it may not be possible to obtain closed formtractable expressions for exact con dence intervals In this context7 asymptotic results are very helpful De nition A random set SX177Xn is an asymptotic con dence set of level 1 7 04 if M0 6 SltXgtgt 1704 asn oo V069 Standard case Suppose that Tn are asymptotically normal WTn 7 0 i N0700 where 00 is the asymptotic variance function For 04 E 071 set lP Z 2 zag 042 the upper 042 quantile Then 5 l Tn70 72042 l OL 172 m ilt gt asngtoo As a particular example7 note that for a maximum likelihood estimator MLE under suitable regularity7 we have shown that 00 10 1721 Variance stabilization The asymptotic variance function 00 can be unpleasant7 since it has a dependence on 0 It is desirable to have asymptotically normal limits in which asymptotic variance 00 E c independent of 0 Look at transformations of Tn with smooth By the delta method Cram r d mm 7 MN a N 0 ltf lt6gtgt2vlt6gt Choose such that f 0 c100 for some constant c Then d This is called variance stabilization since we are choosing a transformation to atten out the variance Example 3 Say X17 7X iid Poi0 Then for the MLE7 W X i s i N070 Hence we want f 0 So let f0 20 with c 1 Then X77 W i N071 17 3 STAT 210A Lecture 17 7 October 26 Fall 2006 Thus 1 7 1 V Xn 7 2 za27 V Xn mzaZ is an asymptotic 1 7 a con dence interval for Since 6 gt 07 this gives a closed form answer for an asymptotic con dence interval for 0 173 Asymptotics for MLE estimators Recall that under suitable regularity7 a sequence of MLEs 0 satis es the asymptotic normality condition we 7 6 i Nan16 where the Fisher information is d 2 EIOEPWMQO 7E 61 logpm 9 19 7 E d62 Hence for MLEs7 as a special case of 1727 IPlt n160n 7 6 2042 717 a In the MLE setting7 there are other ways related to but distinct from direct variance stabilization to deal with the dependence of 0 on 0 First7 if 0 is a continuous function of 07 so that Hag0 i 1 whenever 9 i 07 then n00n 7 a 7 3 n00n 7 9 MPH 3No1 and by Slutsky nI66n 7 0 i N071 STAT 210A Lecture 17 7 October 26 Fall 2006 A more aggressive approximation is we saw this in the proof of ML estimators is given by d2 99n ilEprg lngY gt 0 99 1 d2 g 1093909 9 017 99 JX9nE Sampleobserved Fisher information Conclude 11D xnJX6n6n i 6 za2gta17 a ie 1 1 lt6 nJX6 6 nJX6n gt is asymptotically a 1 7 a con dence interval for the MLE 174 Asymptotic ef ciency and 1step improvements Recall that under regularity we have yaw 7 a i N0 119 173 so that the MLE is said to be asymptotically ef cient Some sideremarks on asymptotic ef ciency 1 By Crame r Rao certainly no unbiased sequence Tn ie ETn 19 V71 could beat 173 2 Hodge7s counterexample from last class super ef cient at 6 0 However Le Cam showed that super ef ciency can occur on a set of Lebesgue measure zero Conse quently if we restrict to estimators with asymptotic variance functions 00 that are continuous then the convergence 173 is optimal in a precise way Although MLEs are asymptotically ef cient they can be expensive to compute Typi cally we need some kind of iterative methods to nd an approximate MLE Newton s method Consider L6 1 21 logpXt9 Looking for solutions to VL0 0 Newton7s in method generates a sequences of iterates k 01 9W1 901 i v2L6ltkgt 1 VL6k 17 5 STAT 210A Lecture 17 7 October 26 Fall 2006 It can be expensive to compute VZL Method of scoring Aproximate V2L6k by the Fisher information 109k This yields 9 9W 19k 1 VLQQW which a particular type of quasi Newton method Strategy Say we have a cheap77 estimator 1 that is asymptotically normal 7 9 i N07110 with 110 gt 10 ie 1 is not asymptotically ef cient Might think about improving 9 by applying a Newton stepmethod of scoring 71 91 on 7 mm VL6n The remarkable thing is that with 1 Newton iteration7 this is all that is needed One step of Newton7s method is suf cient to convert any consistent estimator into an asymptotically ef cient estimator