Stat Methods and Models
Stat Methods and Models STA 251
Popular in Course
Popular in Statistics
This 28 page Class Notes was uploaded by Carmen Mayer on Tuesday September 8, 2015. The Class Notes belongs to STA 251 at University of California - Davis taught by Peter Hall in Fall. Since its upload, it has received 77 views. For similar materials see /class/191913/sta-251-university-of-california-davis in Statistics at University of California - Davis.
Reviews for Stat Methods and Models
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 09/08/15
We wish to prove Theorem Let X X1 X2 denote independent and identically distributed ran dom variables for which ElX s lt 00 for some 5 2 3 0 EX2 l and Crame r s smoothness condition holds limsup lElt6itXl lt l C Mace Then the distribution of 5 E n lZ 2139 X j admits an Edgeworth expansion up to a remainder of smaller order than order n 5 22 PSn S x Mac nil2 R1ac n 5 22 R52 0n 5 22 uniformly in at as n a 00 Note that condition C is equivalent to for each 6 gt 0 sup EeitX lt l 0 ltlgte This follows from the properties condition C implies that the distribution of X is nonlattice and ii any distribution for which the absolute value of the characteristic function equals 1 at a nonzero point is a lattice distribution Proof of Theorem 1 The main ingredient is Esseen7s smoothing lemma Lemma Let F be a nondecreasing function and G a function of bounded variation and write XF and XG for the characteristic functions Fourier Stieltjes transforms of F and G If a F7oo X700 and Foo Goo b G eXists everywhere and WW is uniformly bounded and c f lF 7 Cl lt 00 then for each T gt 0 703 W e cw M dt supwaw where A denotes an absolute constant The niteness of moments of order 5 implies that my 7 ltitjEXj ms lc 10 j where here and below 6k denotes a function satisfying 6kt a 0 as t a 0 It follows that Z 5 it j H39 s 1ogEltetXgte Z m 620 32 and hence that log E6itsn n log EeitX l2 satis es 5 t 12 j V s lowers n 2 n ItWI mm m 12 where here and below 6k satisfies sup sup l6ktnl a 0 as e a 00 7121 ltlgenlQ In View of l 5 12 39 Ewing exp n n ltn12561tn 32 2 572 6 2 1 Z 71772 r3015 n ltn12562tn exp 630371 152 J 1 lt2 Now we apply the smoothing lemma with T n5 and T1 67112 for E gt 0 chosen small but fixed Let F and G equal respectively the distribution function of S7 and am cm nil2 R1 MHW R52x then Xpt Eeitsn 2 572 ms 1 Z n jwa j1 and 01 E supn c sup lGl lt 00 Hence by the smoothing lemma and for large n l sup 7 S 711 2 A01 7175 3 7ltXgtltmltoo 77 where for T1 67112 and T n5 ltl T1 Mien I2 mow mow T1ltltl T 2 Given 6 E 0 i choose 6 gt 0 so small that l62tnl l63tnl S 6 for ltl 3 T1 Then result 2 implies that 12 26 677 hfggfn4amp4VA HP Hmpi76 dtb6n m n Where 02 27r ftgt0ltl571 exp7t2 dt lf ltl gt T1 67112 and X denotes the characteristic function of X then in View of condition C lXFtl Xt 12ln 38713 MW p 5 say Where 0 lt p lt 1 Moreover WMWSOMLHWW WJ SOMW uniformly in all t and so lxgtl S 04 e gW 1 When ltl gt T1 Together 5 and 6 imply that 12 000 7 for each C gt 0 Combining this property With 3 4 and 7 and noting that 6 in 4 can be chosen arbitrarily small we deduce that Sup We Gltmgtl 0n 522 ioltgtltmltltxgt Which implies the theorem METHODOLOGY AND THEORY FOR THE BOOTSTRAP Fifth set of two lectures Main topic of these lectures Completion of work on confidence intervals and sur vey of miscellaneous topics Revision of confidence intervals Recall that a is the a Ievel quantile of the bootstrap distribution of T n12 6 PT alxa A one sided percentile t confidence interval for an unknown parameter 0 based on the bootstrap estimator g and having nominal cov erage 04 is therefore J1 J1a oo n 126 1a Revision continued It has coverage error 001 1 P9 e j1oz or 0n 1 A conventional two sided interval for which the nominal coverage is also or is obtained from two one sided intervals i201 J11ail1 a l 125 1a2 71 12 Wow2 Unsurprisingly the actual coverage of jg also equals or 0n 1 P9 e 301 or 0n 1 Percentile method intervals Interestingly however this result extends to the case of two sided percentile intervals the one sided versions of which have coverage ac curacy only 0n12 Recall that one form of percentile method confidence interval for 6 is f12a oo n 128E1a where Ea is the a level critical point of the bootstrap distribution of sa Psa s Ea l 2c a The corresponding two sided interval is B201 112 1 04 f12 1 04 A 1 2 A A Z l9 n 1a27 91 quot 1255lt1 agt2gt coverage Of two sided percentile intervals To calculate the coverage of f22a recall that 139 E fi2a 04 71 12 P1Zoz Q1ltZ0z 0305 001 1 Since P1 and Q1 are even polynomials and Z1a2 z1a2 then P1Z1a2 Q1Z1Oz2 P1Z1a2 Q1Z1 a2 W herefore 139 E f 2a P 9 E 12 P 9 E 12 1 1 oz 1 O 0nl 04 001 1 Coverage of two sided percentile intervals continued Therefore owing to the parity properties of polynomials in Edgeworth expansions this two sided percentile confidence interval has cover age error On 1 The same result holds true for the other type of percentile confidence interval of which the one sided form is 21a oo n 1Qa a Its one and two sided forms have coverage madam a0n12 Maegan or I On1 Exercise 1 Derive the latter property 2 Show that when computing percentile confidence intervals as distinct from percen tile t intervals we do not actually need the value of a It has been included for didactic reasons to clarify our presentation of theory but it cancels in numerical calculations Discussion Therefore the arguments in favour of percen tile t methods are less powerful when applied to two sided confidence intervals However the asymmetry of percentile intervals will usu ally not accurately reflect that of the statistic and in this sense they are less appropriate This is especially true in the case of the in tervals f the other percentile method There when has a markedly asymmetric distribution the lengths of the two sides of a two sided interval based on R will reflect the exact opposite of the tailweights Other bootstrap confidence intervals It is possible to correct bootstrap confidence intervals for skewness without Studentising The best known examples of this type are the accelerated bias corrected intervals pro posed by Bradley Efron based on explicit cor rections for skewness It is also possible to construct bootstrap con fidence intervals that are optimised for length for a given level of coverage The coverage accuracy of bootstrap confi dence intervals can be reduced by using the iterated bootstrap to estimate coverage error and then adjust for it Each application gen erally reduces coverage error by a factor of 71 12 in the one sided case and 71 1 in the two sided case Usually however only one application is computationally feasible Other bootstrap confidence intervals cont Although the percentile t approach has obvi ous advantages these may not be realised in practice in the case of small samples This is because bootstrapping the Studentised ratio involves simulating the ratio of two random variables and unless sample size is sufficiently large to ensure reasonably low variability of the denominator in this expression poor cov erage accuracy can result Note too that percentile t confidence intervals are not transformation invariant whereas in tervals based on the percentile method are From some viewpoints particularly that of good coverage performance in a very wide range of settings an analogue of robust ness the most satisfactory approach is the coverage corrected form using the iterated bootstrap of first type of percentile method interval ie of in and 32 in one and two sided cases respectively Bootstrap methods fOI time series There are two basic approaches in the time series case applicable with or without a struc tural model respectively We shall say that we have a structural model for a time series X1 Xn if there is a con tinuous deterministic method for generating the series from a sequence of independent and identically distributed disturbances 6162 The method should depend on a finite number of unknown but estimable parame ters Moreover it should be possible to es timate all but a bounded number of the dis turbances from n consecutive observations of the time series Bootstrap fOI time series With structural model We call the model structural because the pa rameters describe only the structure of the way in which the disturbances drive the pro cess In particular no assumptions are made about the disturbances apart from standard moment conditions In this sense the setting is nonparametric rather than parametric The best known examples of structural mod els are those related to linear time series for example the moving average 19 Xj M 9 j 1 11 or an autoregression such as p Xj MZ Z wz Xj z 1 M ja i1 where u 619p w1wp and perhaps also p are parameters that have to be es timated Bootstrap for time series with structural model continued 1 In this setting the usual bootstrap approach to inference is as follows 1 Estimate the parameters of the structural model eg u and w1wp in the autore gression example and compute the residuals ie estimates of the ej39S using standard methods for time series 2 Generate the estimated time series in which true parameter values are replaced by their estimates and the disturbances are re sampled from among the estimated ones ob taining a bootstrapped time series X X for example in the autoregressive case p X X1 m e Z Bootstrap for time series with structural model continued 1 3 Conduct inference in the standard way using the resample XX thus obtained For example to construct a percentile t con fidence interval for u in the autoregressive ex ample let 32 be a conventional time series estimator of the variance of 71121 computed from the data X1 Xn let if and 62 de note the versions of pi and 32 computed from the resampled data XX and construct the percentile t interval based on using the bootstrap distribution of as an approximation to the distribution of T 711207 loa Bootstrap for time series with structural model continued 2 All the standard properties we have already noted founded on Edgeworth expansions ap ply without change provided the time series is sufficiently short range dependent Early work on theory in the structural time series case includes that of Bose A 1988 Edgeworth correction by bootstrap in autoregressions Ann Statist 16 1709 1722 Bootstrap for time series with structural model continued 3 It is common in this setting not to be able to estimate n disturbances ej based on a time series of length n For example in the context of autoregressions we can generally estimate no more than n p of the distur bances But this does not hinder application of the method we merely resample from a set of n p rather than n values of E7 Usually it is assumed that the disturbances have zero mean We reflect this property em pirically by centring the Ej s at their sample mean before resampling Bootstrap fOI time series WithOUt StI UC tural model In some cases for example where highly non linear filters have been applied during the pro cess of recording data it is not possible or not convenient to work with a structural model There is a variety of bootstrap methods for conducting inference in this setting based on block or sampling window methods We shall discuss only the block bootstrap ap proach BIOCK bootstrap fOI time series Just as in the case of a structural time series the block bootstrap aims to construct simu lated versions of the time series which can then be used for inference in a conventional way The method involves sampling blocks of con secutive values of the time series say XI1 X1b where 0 g I g n b is chosen in some random way and placing them one af ter the other in an attempt to reproduce the series Here b denotes block length Assume we can generated blocks X1j1 X1jb for j 2 1 ad infinitum in this way Cre ate a new time series XX identical to X1117 7X11b7 X1217 7X12b7 The resample X X isjust the first n val ues in this sequence Block bootstrap for time series contin ued 1 There is a range of methods for choosing the blocks One the fixed block approach in volves dividing the series X1 Xn up into m blocks of b consecutive data assuming n bm and choosing the resampled blocks at random In this case the Ij s are indepen dent and uniformly distributed on the values 1b 1m 1b 1 The blocks in the fixed block bootstrap do not overlap Another the moving blocks technique al lows block overlap to occur Here the Ij s are independent and uniformly distributed on the values O1n b Block bootstrap for time series contin ued 2 In this way the block bootstrap attempts to preserve exactly within each block the de pendence structure of the original time se ries X1Xn However dependence is cor rupted at the places where blocks join Therefore we expect optimal block length to increase with strength of dependence of the time series Techniques have been suggested for match ing blocks more effectively at their ends for example by using a Markovian model for the time series This is sometimes referred to as the matched block bootstrap Difficulties with the blOCk bootstrap The main problem with the block bootstrap is that the block length b which is a form of smoothing parameter needs to be chosen Using too small a value of b will corrupt the dependence structure increasing the bias of the bootstrap method and choosing b too large will give a method which has relatively high variance and consequent inaccuracy Another difficulty is that the percentile t ap proach cannot be applied in the usual way with the block bootstrap if it is to enjoy high levels of accuracy This is because the corrup tion of dependence at places where adjacent blocks joins significantly affects the relation ship between the numerator and the denomi nator in the Studentised ratio with the result that the block bootstrap does not effectively capture skewness However there are ways of removing this problem Successes Of the DIOCK bootstrap Nevertheless the block bootstrap and related methods give good performance in a range of problems where no other techniques work effectively for example inference for certain sorts of nonlinear time series The block bootstrap also has been shown to work effectively with spatial data There the blocks are sometimes referred to as tiles and either of the fixed block or moving block methods can be used References for DIOCK bootstrap Carlstein E 1986 The use of subseries values for estimating the variance of a gen eral statistic from a stationary sequence Ann Statist 14 1171 1179 Hall P 1985 Resampling a coverage pat tern Stochastic Process Appl 20 231 246 KUnsch H R 1989 The jackknife and the bootstrap for general stationary observations Ann Statist 17 1217 1241 Politis DN Romano JP Wolf M 1999 Subsampling Springer New York Bootstrap in non regular cases There is a meta theorem which states that the standard bootstrap which involves con structing a resample that is of approximately the same size as the original sample works in the sense of consistently estimating the lim iting distribution of a statistic if and only if that statistic s distribution is asymptotically Normal It does not seem possible to formulate this as a general rigorously provable result but it nevertheless appears to be true The result underpins our discussion of boot strap confidence regions which has focused on the case where the statistic is asymptoti cally Normal Therefore rather than take up the issue of whether the bootstrap estimate of the statistic s distribution is asymptotically Normal we have addressed the problem of the size of coverage error Example of non regular cases Perhaps the simplest example where this ap proach fails is that of approximating the dis tributions of extreme values To appreciate why there is difficulty consider the problem of approximating the joint distribution of the two largest values of a sample X1Xn from a continuous distribution The probabil ity that the two largest values in a resample Xi X drawn by sampling with replace ment from the sample both equal maxX is 1 n l 1 1 n l 1 1 n 1 gt1 2e1 TL TL TL aSn gtOO The fact that the probability does not con verge to zero makes it clear that the joint dis tribution of the two largest values in the boot strap sample cannot consistently estimate the joint distribution of the two largest data The m Out Of n bootstrap The most commonly used approach to over coming this difficulty in the extreme value ex ample and many other cases is the m out of n bootstrap Here rather than draw a sam ple of size n we draw a sample of size m lt n and compute the distribution approximation in that case Provided mmn gtoo and mn gtO the m out of n bootstrap gives consistent es timation in most probably all settings For example this approach can be used to consistently approximate the distribution of the mean of a sample drawn from a very heavy tailed distribution for example one in the do main of attraction of a non Normal stable law The m out of n bootstrap continued The main difficulty with the m out of n bootstrap is choosing the value of m Like block length in the case of the block boot strap m is a smoothing parameter large m gives low variance but high bias and small m has the opposite effect In most problems where we would wish to apply the m out of n bootstrap it proves to be quite sensitive to selection of m A secondary difficulty is that the accuracy of m out of n bootstrap approximations is not always good even if m is chosen optimally For example when the m out of n bootstrap is applied to distribution approximation prob lems the error is often of order m12 which since mn gt O is an order of magnitude worse than 71 12
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'