### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# Mathematical Statistics STAT 709

UW

GPA 3.57

### View Full Document

## 17

## 0

## Popular in Course

## Popular in Statistics

This 137 page Class Notes was uploaded by Mrs. Triston Collier on Thursday September 17, 2015. The Class Notes belongs to STAT 709 at University of Wisconsin - Madison taught by Jun Shao in Fall. Since its upload, it has received 17 views. For similar materials see /class/205096/stat-709-university-of-wisconsin-madison in Statistics at University of Wisconsin - Madison.

## Reviews for Mathematical Statistics

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 09/17/15

Stat 709 Mathematical Statistics Lecture 38 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA s s 3333 Theorem 311 Consistency peat c Consider model X Z 3 s 1 under assumption A3 Es O and Var is an unknown matrix Consider the LSE 73 with l e Z for every n Suppose that sup MVars lt co where 1A is the largest eigenvalue of the matrix A and that limnsw MZTZ 0 Then FE is consistent in mse for any I e Z Lure s v poo Theorem 311 Consistency Consider model X Z 3 s 1 under assumption A3 Es O and Var is an unknown matrix Consider the LSE TB with l e Z for every n Suppose that sup MVars lt co where 1A is the largest eigenvalue of the matrix A and that limnsw MZTZ 0 Then TB is consistent in mse for any I e Z The result follows from the fact that FE is unbiased and VarITB T272zr Var ZZTZ I 1Var TZTZ I Without the normality assumption on s the exact distribution of 73 is very hard to obtain The asymptotic distribution of TB is derived in the following result Without the normality assumption on s the exact distribution of 73 is very hard to obtain The asymptotic distribution of TB is derived in the following result Consider model 1 with assumption A3 Suppose that O lt inf 7LVars where L A is the smallest eigenvalue of the matrix A and that 39 T T i 7 mmrggz Z Z Z 70 2 Suppose further that n 2L1 m for some integers k mjj1k with mj s bounded by a fixed integer m s 515k 516 mx and s are independent i If supElsl25 lt co then for any I e Z 7Ei VarI73Hd NO1 3 ii Result 3 holds for any I 692 if when m m 1 g i ltj g k 5 and 5 have the same distribution i For I 62 7272 ZTZB 7713 O and k WM 7 mzizrzfs 7 2 cat I where cnj is the mjvector whose components are ITZTZ Z i Ig11kjk0 0 and kj 211 mtj 1k Note that k 2 HemH2 7 ITszszszI 7 rm 4 j1 Also 2 lt 7 7 7 2 12 Honii 7 mfggil Z Z 2 lt 7 7 7 j 7 7 iml Z Z 1212572 Z Z Z which together with 4 and condition 2 implies that Proof continued k 39 2 2 0 nmlt1gikacmH A HonH gt The results then follow from Corollary 13 Proof continued k 39 2 2 0 m gallonll A lienll gt nape The results then follow from Corollary 13 0 Under the conditions of Theorem 312 Var is a diagonal block matrix with Var j as the jth diagonal block which includes the case of independent 8 s as a special case 0 Exercise 80 shows that condition 2 is almost a necessary condition for the consistency of the LSE Proof continued 3 k 2 2 gmlt1rgixkllcmll jllcmll gt 0 The results then follow from Corollary 13 0 Under the conditions of Theorem 312 Var is a diagonal block matrix with Var j as the jth diagonal block which includes the case of independent 8 s as a special case 0 Exercise 80 shows that condition 2 is almost a necessary condition for the consistency of the LSE The following are sufficient conditions for 2 a 1ZTZ a O and ZfZTZ Z7 a O as n a co b There is an increasing sequence an such that an a co awnan a 1 and ZTZan converges to a positive definite matrix Proof of a Since 272 depends on n we denote ZTZ by A Let in be the integer such that h max1g n h lf limnin co then Iimhi IimeAnZ limeAIHZ o n n n where the inequality follows from in g n and thus A 7A is nonnegative definite Proof of a Since 272 depends on n we denote ZTZ by A Let in be the integer such that h max1 ign h lf limnin co then Iimhi IimeAnZ limeAIHZ o n n n where the inequality follows from in g n and thus A 7A is nonnegative definite If in g c for all n then Iim h Iim zfAnz Iim An max llZIll2 0 n n n 193 Therefore for any subsequence jn C in with limnjn a e Ooltgt im j O This shows that limnh O Example simple linear model In Example 312 Xi O l B1ti l 5i7 i1 If n 1 2771 ti a c and n 12f 1tH d where c is positive and c gt d2 then condition b in Lemma 33 is satisfied with an n and therefore Theorem 312 applies n 7 Example oneway ANOVA r In the oneway ANOVA model Example 313 mpg8 ik11kjj1m where k00kJZJI1nj1m and 1um 7 7 7 7 7 7 7 71 12257242 Z 277LZ Z 71r2jxnnj Conditions related to Z in Theorem 312 are satisfied iff minjnj a co Some similar conclusions can be drawn in the twoway ANOVA model Example 3 14 eghtdLs In the linear model X Z 3 s the unbiased LSE of TB may be improved by a slightly biased estimator when V Var is not 72 and the LSE is not BLUE In the linear model The wetd LS X 213 s the unbiased LSE of TB may be improved by a slightly biased estimator when V Var is not 72 and the LSE is not BLUE Assume that Z is of full rank so that every TB is estimable If V is known then the BLUE of TB is mi where E zrvi1zr1zrv4x see the discussion after the statement of assumption A3 in 331 5 The weighted LSE In the linear model X Z 3 s the unbiased LSE of TB may be improved by a slightly biased estimator when V Var is not 72 and the LSE is not BLUE Assume that Z is of full rank so that every TB is estimable If V is known then the BLUE of TB is 713 where EZTV 1Z 1ZTV 1X 5 see the discussion after the statement of assumption A3 in 331 If V is unknown and V is an estimator of V then an application of the substitution principle leads to a weighted least squares estimator 3W ZTV 1Z 1ZTV 1X 6 The weighted LSE is not linear in X and not necessarily unbiased for B If the distribution of s is symmetric about 0 and V remains unchanged when 8 changes to is then the distribution of 3w 7 B is symmetric about 0 and if EBW is well defined 3W is unbiased for B Ifthe weighted LSE FEW is unbiased then the LSE FE may not be a BLUE since VarlTBW may be smaller than VarlTB Ifthe weighted LSE FEW is unbiased then the LSE FE may not be a BLUE since VarlTBW may be smaller than VarlTB Asymptotic properties of the weighted LSE depend on the asymptotic behavior of We say that V is consistent for V iff HVAV InHmaX H1307 7 where HAHmaX max iaUi for a matrix A whose ijth element is a Ifthe weighted LSE FEW is unbiased then the LSE FE may not be a BLUE since VarlTBW may be smaller than VarlTB Asymptotic properties of the weighted LSE depend on the asymptotic behavior of We say that V is consistent for V iff HVAV InHmaX H1307 7 where HAHmaX max iaUi for a matrix A whose ijth element is 3 Theorem 317 Consider model 1 with a fu rank Z LetE and EM be defined by 5 and 6 respectively with a V consistent in the sense of 7 Underthe conditions in Theorem 312 3w iman Ha N0717 where l e 3 ly O and a2 Varl7 ITZTV 1Z 1I Proof Using the same argument as in the proof of Theorem 312 we obtain that V W3 7Ban ad N071 By Slutsky s theorem the result follows from FEW 773 oa Define A A 5 ITZTV 1Z 1ZTV 1 7 V 1s and A 5 ITZTV 1Z 17ZTV 1Z 1ZTV 15 Then FEW Pg 5n n The result follows from 5 oa and n oa details are in the textbook 0 Theorem 317 shows that as long as V is consistent in the sense of 7 the weighted LSE 3W is asymptotically as efficient as which is the BLUE if V is known 0 By Theorems 312 and 317 the asymptotic relative efficiency of the LSE FE wrt the weighted LSE FEW is ITZTV 1Z 1I ITZTZ1ZTVZZTZ1I whichjs always less than 1 and equals 1 if FE is a BLUE in which case B B 0 Finding a consistent V is possible when V has a certain type of structure Example 329 Consider model 1 Suppose that V Var is a block diagonal matrix with the ith diagonal block a2lmUUf i1k 8 where m s are integers bounded by a fixed integer m 72 gt O is an unknown parameter 2 is a q x q unknown nonnegative definite matrix U is an m x q full rank matrix whose columns are in W q lt infm and W is the px m matrix such thatZT W1 W2 Wk Example 329 l Consider model 1 Suppose that V Var is a block diagonal matrix with the ith diagonal block a2lmUUf i1k 8 where m s are integers bounded by a fixed integer m 72 gt O is an unknown parameter 2 is a q x q unknown nonnegative definite matrix U is an m x q full rank matrix whose columns are in W q lt infm and W is the px m matrix such thatZT W1 W2 Wk Under 8 a consistent V can be obtained if we can obtain consistent estimators of 72 and 2 Let X Y1 Yk where Y is an mvector and let R be the matrix whose columns are linearly independent rows of W Then 2 1 k M 2 Yfllm e RmRfRIWRHYI i1 8 is an unbiased estimator of 72 Example 923 09 winged Assume that Y s are independent and that supEisi25 lt co for some 5 gt 0 Then 82 is consistent for 72 exercise Let r Yii Wf and 1 A Ufui 1 UiTnriTUIUITUIr1 702U7UIr1i I i Itfan be shown exercise that f is consistent for X in the sense that H2 7 Xiimax ap O or equivalently ifiiii ap 0 see Exercise 116 Stat 709 Mathematical Statistics Lecture 8 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 8 Conditional expectation In elementry probability conditional probability PBlA is defined as PBlA PA BPA for events A and B with PA gt O For two random variables X and Y how do we define PX e BlY y gm onall i orm In elementry probability conditional probability PBlA is defined as PBlA PA BPA for events A and B with PA gt O For two random variables X and Y how do we define PX e BlY y D V 70n3916 Let X be an integrable random variable on Q f i The conditional expectation of X given M a subafield of 9 denoted by EXl is the asunique random variable satisfying the following two conditions a EXlsz is measurable from 0527 to Q b fAEXlszdP fA XdP for any A e 527 ii The conditional probability of B e 9 given M is defined to be PBl EUsl l iii Let Y be measurable from Q f to mg The conditional expectation of X given Y is defined to be EXlY EXl6Yl V 5 The existence of EXi follows from Theorem 14 o 0Y contains the information in Yquot o EXiY is the expectation ofX given the information in Y o For a random vector X EXi is defined as the vector of conditional expectations of components of X 0 The existence of EXl follows from Theorem 14 o 0Y contains the information in Yquot o EXlY is the expectation ofX given the information in Y o For a random vector X EXl is defined as the vector of conditional expectations of components of X Let Y be measurable from 19 to A3 and Z a function from 19 to Q Then Z is measurable from QaY to gh k iff there is a measurable function h from A to kk such that Z ho Y O The existence of EXl2 follows from Theorem 14 o 0Y contains the information in Yquot o EXlY is the expectation ofX given the information in Y o For a random vector X EXl2 is defined as the vector of conditional expectations of components of X Lemma 12 Let Y be measurable from 19 to A3 and Z a function from my to 3 Then Z is measurable from QaY to gh k iff there is a measurable function h from A to kk such that Z ho Y By Lemma 12 there is a Borel function h on A such that EXl Y ho Y For y e A we define EXl Y y hy to be the conditional expectation of X given Y y hy is a function on A whereas ho Y 7 EXlY is a function on Q Let X be an integrable random variable on Q f A1A2 be disjoint events on Q f such that UA Q and PAi gt O for all i and let 31612 be distinct real numbers Define Y a1IA1azlA2 We now show that EXiY 2 WP AW i1 PAi We need to verify a and b in Definition 16 with M 0Y Let X be an integrable random variable on Q f A1A2 be disjoint events on Q f such that UA Q and PAi gt O for all i and let 31612 be distinct real numbers Define Y a1lA1azlA2 We now show that w XdP Egtltly21 2 1A We need to verify a and b in Definition 16 with M 0Y Since 0Y 0A1A2 it is clear that the function on the righthand side is measurable on QaY This verifies a Let X be an integrable random variable on Q f A1A2 be disjoint events on Q f such that UA Q and PAi gt O for all i and let 31612 be distinct real numbers Define Y a1lA1azlA2 We now show that w XdP Egtltly21 2 1A We need to verify a and b in Definition 16 with M 0Y Since 0Y 0A1A2 it is clear that the function on the righthand side is measurable on QaY This verifies a To verify b we need to show w A XdP XdP i I dP WB MB PAi A for any B 6 Example 121 continued 1 Using the fact that Y 1 B U alesAi we obtain XdP XdP WB Z iaeB A fAXdP 2 1 PM P Am r1 5 A XdP l I dP yl3 WA A where the last equality follows from Fubini s theorem This verifies b and thus the result Let h be a Borel function on Q satisfying ha A XdPPA Then EXlY hoY and EXlYyhy Proposition 19 Let X be a random nvector and Y a random mvector Suppose that X Y has a joint pdf fxy wrt v x it where v and it are afinite measures on and mm respectively Let gxy be a Borel function on QM quot for which EigX Y lt co Then EgX my f 9fp7f avx as Proposition 19 Let X be a random nvector and Y a random mvector Suppose that X Y has a joint pdf fxy wrt v x it where v and it are afinite measures on and mm respectively Let gxy be a Borel function on QM quot for which EigX Y lt co Then unvvonodwo ffx Ydvx atIIIIIIIIIIIIIIIIIIII Denote the righthand side by hY By Fubini s theorem h is Borel Then by Lemma 12 hY is Borel on QaY Also by Fubini s theorem EgX YiY as mna mnwm isthepdf onwrt7L VProof continued I For B 6 gm h Y dP h dP Y lB B y Y 9Xyfxydvx 7 Wf dlm f d A WX39X7V XMV Vx X dP xB g 7y va X dP WBg 7 where the first and the last equalities follow from Theorem 12 the second and the next to last equalities follow from the definition of h and pdf s and the third equality follows from Fubini s theorem Conditional pdf 39 Let X Y be a random vector with a joint pdf fxy wrt v x it The conditional pdf of X given Y y is defined to be leYXlV fX7VfYV where MY fx7ydvx is the marginal pdf of Y wrt it For each fixed y with fy y gt O fXMxly is a pdf wrt v Then Proposition 19 states that Eigrxwvi gx7 YfxlyXlYdvX ie the conditional expectation of gX Y given Y is equal to the expectation of gX Y wrt the conditional pdf ofX given Y i Let X Y X1X2 be integrable random variables on Q f and M be a subafield of 9 i le c as c 6 then EXlng c as ii le3 Y as then EXl g EYl as iii If ab 6 Q then EaXbYl aEXl bEYl as iv EEXlg2f EX v EEXl ll E subafield of M vi If 0Y C Mand ElXYl lt co then EXYlng YEXlng as vii le and Y are independent and ElgX Yl lt co for a Borel function 9 then EgX YlY y EgXy as Py viii If EX2 lt co then EXl 2 g EX2l as ix Fatou s lemma If X 2 O for any n then E liminannlM g liminanXlng as x Dominated convergence theorem If anl g Y for any n and X a X then Ean a EXl XlMO EEXll as where Mo is a Example 122 i Let X be a random variable on Q f with EX2 lt co and let Y be a measurable function from REQP to mg One may wish to predict the value of X based on an observed value of Y Let gY be a predictor ie g e N all Borel functions 9 with EgY2 lt co Each predictor is assessed by the mean squared prediction errorquot ElXgYl2 We now show that EXl Y is the best predictor of X in the sense that EX7EXlY2 migEXigY2 96 First Proposition 110viii implies EXlY e N Example 122 continued 1 Next for any 9 e N EXltCIY2 EX45leEle79y2 EXEle2ElEle7900 2EH 5XlYEXlY9 Y EXEle2EEXlY790012 2 5EX45XlYEXlY 1YlY EXEle2EEXlY790012 2EEXlY790015 EWYWH EXEle2EEXlY790012 ZEXEXlY27 2 where the third equality follows from Proposition 110iv the fourth equality follows from Proposition 110vi and the last equality follows from Proposition 110i iii and vi Stat 709 Mathematical Statistics Lecture 11 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 11 Convergence modes and stochastic orders c 01quot ck e k HcH 211 telV r gt 0 If r 2 1 then HcH is the Lrdistance between 0 and c When r2 HcH Hon xCTC c c1cke zk llcllr2f1l6jl 1 yr gt 0 If r 2 1 then Hell is the Ldistance between 0 and c When r2 llcll llcllz xCTC Let XX1X2 be random kvectors defined on a probability space i We say that the sequence X converges to X almost surely as and write X Ha X iff limnHan X as ii We say that Xn converges to X in probability and write X Hp X iff for every fixed 8 gt 0 Jim merxii gt s o anytime w 1mm Definition 18 continued iii We say that X converges to X in L or in rth moment and write X AL X iff Iim ElaniXlll o where r gt O is a fixed constant iv Let F F n 12 be cdf s on 3quot and P P n 1 be their corresponding probability measures We say that Fn converges to F weakly or Pn converges to P weakly and write F aw F or P7 AW P iff for each continuity point X of F IliincFMX FX We say that X converges to X in distribution or in law and write Xn ad X iff Fx Hw FX 0 Has Hp ab How close is between X and X as n a co 0 FX aw FX FXH is close to FX but X and X may not be close they may be on different spaces 0 Has Hp ab How close is between X and X as n a co 0 FX aw FX FXH is close to FX but X and X may not be close they may be on different spaces Example 126 Let 9 1 n 1 and X be a random variable having the exponential distribution EO 97 Table 12 n 12 Let X be a random variable having the exponential distribution EO1 Foranyx gt0 asnaoo FXHX17e X9 H17e x FXX Since FXHX E O E FXX for X g 0 we have shown that XanX Xn Hp X 0 Need further information about the random variables X and X 0 We consider two cases in which different answers can be obtained X Hp X 0 Need further information about the random variables X and X 0 We consider two cases in which different answers can be obtained Case 1 Suppose that X E 9X then X has the given cdf XniX 9771X n 1Xwhich has the cdf 1einXowx Then X Hp X because for any 8 gt O PiX7Xi ZS e 75 0 In fact by Theorem 18v X Ha X Also Xn ALF X for any p gt 0 because EaniXi n PEX s o i Suppose that X and X are independent random variables Since pdf s for X and 7X are 971e X9 l07w X and eXIw70X respectively we have 8 PanXl s L e e Xe ewosxI7wxydxdy which a to bythe39 39 39 theorem 5 IeixeyixlmywXlwyxydxdy 1 eis39 Thus Paxrxi 8 He s gt0 for any 8 gt O and therefore X Hp X does not hold Proposition 11 6 Polya s theorem If F7 aw F and F is continuous on k then Iim sup iFX7FXi O quotwae k Proposition 11 6 Polya s theorem If F aw F and F is continuous on k then lim sup lFX7FXl O nimxe k This proposition implies the following useful result If F aw a continuous F and on 6quot with 0 a c then Fnc a Fc Proposition 116 Polya s theorem E If F aw F and F is continuous on k then lim sup lFX7FXl O nimxe k This proposition implies the following useful result If F aw a continuous F and on 6quot with 0 a c then Fnc a Fc For ranomkvectors XX1X27 on a probabty space X a3 X iff for every 8 gt 0 map lt O llxmixll gt 8gt o Proof It can be verified that AjwiigannwXw7 AI U ixm XHSJq j1 n1mn By Proposition 11iii continuity PAj JWMPltQ iinrXH Sf mn 17JmPltO HXm XH gtj 1gt PUniiXmiXH gt 8 a O for every 8 gt 0 iff PAj 1 for everyj which is equivalent to P j 1Aj 1 Le Xn Ha X because PAjPlt AJgt 17PCOAJCgt 217PAf 391 391 J Lemma 15 BoreICantelli lemma Let An be a sequence of events in a probability space and IimsupAn U Am 7 n1mn i If 2 PAn lt co then Pim supnAn 0 ii If A1A2 are pairwise independent and 21PAn co then Pim supnAn 1 Lemma 15 BorelCantelli lemma Let An be a sequence of events in a probability space and limsupAn U Am 7 n1mn i If 2 PAn lt co then Pim supnAn 0 ii If A1A2 are pairwise independent and 21PAn co then Pim supnAn 1 Proof of Lemma 15 i By Proposition 11 P limsupAngt lim P lt U Am 3 lim 2 PAn 0 n m aw m7 nawmn in where the last equality follows from the condition i PAn lt co n1 Proof of mea 15 ii i We prove the case of independent An s See Chung 1974 pp 7678 forthe pairwise independence An s P limsupAngt Iiim P lt U Am 1 iliim P lt Afngt aw Hm 7 Hm mn min nk nk nk nk 1 PWquot 11 4mm 11 expePAmexpe 2 WW 1 it g equot expt Letting k gt co m nk w H PWquot gig H PWquot exp7 2 PAm 0 Hence 5wa lt 3 A3 Jm f1 PAf 0 quot1quot The notion of O o and stochastic O and o In calculus two sequences of real numbers an and bn satisfy 0 an Obn iff lanl g clbnl for all n and a constant c 0 an ob iff awnbn 0 as Dace In calculus two sequences of real numbers an and bn satisfy 0 an 0bn iff lanl g clbnl for all n and a constant c 0 an ob iff anbn 0 as new Definition 19 Let X1X2 be random vectors and Y1 Y2 be random variables defined on a common probability space i X OY as iffPllXll OlYl 1 ii Xn oY as iff XnYn H150 iii Xn OpYn iff for any 8 gt 0 there is a constant C5 gt 0 such that supPlanll 2 Clenl lt 2 I7 iv X oy ifanYn HP 0 i 0 Since an 01 means that an is bounded X is said to be bounded in probability if X Op1 O X opY implies Xn OpYn 0 X OpYn and Y OpZn implies Xn OpZn 0 X OpYn does not imply Yn OpXn o If X OpZn then XY7 OpYnZn o If X OpZn and Y OpZn then X Yn OpZn 0 The same conclusion can be obtained if Op and op are replaced by 0 as and o as respectively 0 If X ad X for a random variable X then X Op1 o If Eanl Oa then X Opa where an e Ooltgt o If X Ha X then supn anl Op1 Stat 709 Mathematical Statistics Lecture 20 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 20 Minimal sufficiency Maximal reduction without loss of infortion 7 0 There are many sufficient statistics for a given family 9 0 In fact X the whole data set is sufficient 0 If T is a sufficient statistic and T 1118 where w is measurable and S is another statistic then S is sufficient 0 This is obvious from Theorem 22 if the population has a pdf but it can be proved directly from Definition 24 Exercise 25 0 For instance if X1X7 are iid with PX 1 9 and PX O17 9 then 2511M 2Fm1 is sufficient for 9 where m is any fixed integer between 1 and n o If T is sufficient and T 1118 with a measurable function wthat is not onetoone then 0T C 78 and T is more useful than 8 since T provides a further reduction of the data or afield without loss of information o Is there a sufficient statistics that provides quotmaximalquot reduction of the data Minimal sufficiency V istatement holds except for outcomes in an event A satisfying PA O for all P e 9 then we say that the statement holds as 9 Minimal sufficiency Convention If a statement holds except for outcomes in an event A satisfying PA O for all P e 9 then we say that the statement holds as 9 Definition 25 Minimal sufficiency Let T be a sufficient statistic for P e 9 T is called a minimal suf cient Statistic iff for any other statistic 8 sufficient for P e 9 there is a measurable function 11 such that T 1118 as 9 Minimal sufficiency If a statement holds except for outcomes in an event A satisfying PA O for all P e 9 then we say that the statement holds as 9 Definition 25 Minimal sufficiency Let T be a sufficient statistic for P e 9 T is called a minimal suf cient Statistic iff for any other statistic 8 sufficient for P e 9 there is a measurable function 11 such that T 1118 as 9 Minimal sufficient statistics exist under weak assumptions eg 9 contains distributions on 9quot dominated by a afinite measure Bahadur 1957 If both T and S are minimal sufficient statistics then by definition there is onetoone measurable function 11 such that T 1118 as 9 Hence the minimal sufficient statistic is unique in the sense that two statistics that are onetoone measurable functions of each other can be treated as one statistic 1 If both T and S are minimal sufficient statistics then by definition there is onetoone measurable function 11 such that T 1118 as 9 Hence the minimal sufficient statistic is unique in the sense that two statistics that are onetoone measurable functions of each other can be treated as one statistic Example 213 Let X1X7 be iid random variables form P9 the uniform distribution U991 9 e R ngt 1 The joint Lebesgue pdf of X1X is n f9X 1II9791X39IXW17XW97 X X17Xn E Q 1 where X denotes the ith smallest value of X1 X By Theorem 22 T X1X is sufficient for 9 Example 213 continued We now show that T X1X is minimal sufficient Note that X1 sup9 f9x gt O and X 1 inf9 f9x gt 0 If SX is a statistic sufficient for 9 then by Theorem 22 there are Borel functions h and ge such that f9 X 99SXhx For X with hx gt O X1 sup9 99SX gt O and X 1 inf9 99SX gt 0 Hence there is a measurable function wsuch that TX wSX when hx gt 0 Since h gt O as 9 we conclude that T is minimal sufficient Example 213 continued We now show that T X1X is minimal sufficient Note that X1 sup9 f9x gt O and X 1 inf9 f9x gt 0 If SX is a statistic sufficient for 9 then by Theorem 22 there are Borel functions h and ge such that f9 X 99SXhX For X with hx gt O X1 sup9 99SX gt O and X 1 inf9 99SX gt 0 Hence there is a measurable function wsuch that TX wSX when hx gt 0 Since h gt O as 9 we conclude that T is minimal sufficient Useful tools for finding minimal sufficient statistics Finding a minimal sufficient statistic by definition is not convenient The next theorem is a useful tool Theorem 23 Let 9 be a family of distributions on 9quot i 3 A Suppose that 90 C 9 and as 90 implies as 9 If T is sufficient for P e 9 and minimal sufficient for P e 90 then T is minimal sufficient for P e 9 Suppose that 9 contains pdf s f0f1f2 wrt a afinite measure Let fwx 20 cfX where c gt O for all i and 20 0 1 and let TX fXfxx when fwx gt O i 012 Then TX T0 T1 T2 is minimal sufficient for P e 9 Furthermore if X fX gt O C X f0x gt O for all i then we may replace fwx by f0x in which case TX T1 T2 is minimal sufficient for P e 9 Suppose that 9 contains pdf s fp wrt a afinite measure and that there exists a sufficient statistic TX such that for any possible values X and y of X fpx fpy xy for all P implies TX Ty where d is a measurable function Then TX is minimal sufficient for P e 9 Proof i If S is sufficient for P e 9 then it is also sufficient for P e 90 and therefore T 1118 as 90 holds for a measurable function 11 The result follows from the assumption that as 90 implies as 9 ii Note that foo gt O as 9 Let gT T i012 Then fX gTXfwx as 9 By Theorem 22 T is sufficient for P e 9 Suppose that SX is another sufficient statistic By Theorem 22 there are Borel functions h and g such that fx Sxhx i 012 Then Tix isx 29mm for X s satisfying fwx gt 0 By Definition 25 T is minimal sufficient for P e 9 The proof for the case where foo is replaced by f0 is the same i iii From Bahadur 1957 there is a minimal sufficient statistic SX The result follows if we can show that TX wSX as 9 for a measurable function 11 By Theorem 22 there are Borel functions h and 9 such that fpX gpSXhX for all P Let A X hX 0 Then PA O for all P For X and y such that SX Sy X A and y A fpX gPSXhX 9P3Vhx fPVhXhV for all P Hence TX Ty This shows that there is a function 11 such that TX wSX except for X e A It remains to show that w is measurable Since Sis minimal sufficient gTX SX as 9 for a measurable function 9 Hence 9 is onetoone and w 9 1 By Theorem 39 in Parthasarathy 1967 w is measurable Example 214 Let 9 f9 9 e e be an exponential family with pdf s feX eXPW9TTX 759hx Suppose that there exists 60 90 91 9p C e such that the vectors 71 n9 77190 i 1 p are linearly independent in 3 This is true if the family is of full rank We have shown that TX is sufficient for 9 e e We now show that T is in fact minimal sufficient for 9 e e Let 90 fei 9 E 90 Note that the set X f9 X gt 0 does not depend on 9 It follows from Theorem 23ii with foo f9U that 00 ltexpniTX75177expn TX5pgt is minimal sufficient for 9 e 60 Since his are linearly independent there is a onetoone measurable function 11 such that TX wSX as 90 Hence T is minimal sufficient for 9 e 60 It is easy to see that as 90 implies as 9 Thus by Theorem 23i T is minimal sufficient for 9 e e Exercise The results in Example 213 and 214 can also be proved by using Theorem 23iii J Exercise The results in Example 213 and 214 can also be proved by using Theorem 23iii 39 Stat 709 Mathematical Statistics Lecture 1 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Chapter 1 Probability Theory Lecture 1 Measurable space measure and probability Random experiment uncertainty in outcomes J Chapter 1 Probability Theory Lecture 1 Measurable space measure and probability Random experiment uncertainty in outcomes Sample space Q sample or outcome space a set containing all possible outcomes if ii 9W ll eeg and 1 Random experiment uncertainty in outcomes Sample space Q sample or outcome space a set containing all possible outcomes be a collection of subsets of a sample space Q 9 is called a afield or aalgebra iff it has the following properties i The empty set 0 e 9 ii lfA e 9 then the complement A e 9 iii lfA e 9 i 12 then their union UA e 9 19 is a measurable space if 9 is a afield on Q lhap ter Plrdoaioillity Thea L admire r f lirileersw 7 I Q relate mergeulre lpjrcmbalm ll iiy Random experiment uncertainty in outcomes l Sample space 7 Q sample or outcome space a set containing all possible outcomes t AM i Let 9 be a collection of subsets of a sample space Q 9 is called a afield or aalgebra iff it has the following properties i The empty set 0 e 9 ii lfA e 9 then the complement A e 9 iii lfA e 9 i 12 then their union UA e 9 Q is a measurable space if 9 is a afield on Q 9 is a collection set of sets Two trivial examples 9 contains 2 and Q only 9 contains all subsets of Q power set Wit Why do we need to consider other ofield We may be interested in a particular collection of sets eg 9 0AACQ where A C Q 6 a collection of sets of interest 6 may not be a afield off the smallest afield containing 6 the afield generated by 6 766 6 if 6 itself is a afield 39 isaafieldonQandcg C Cg yer am 6A7A comm ammo Mime We may be interested in a particular collection of sets eg 9 0AACQ where A C Q 6 a collection of sets of interest 6 may not be a afield off the smallest afield containing 6 the afield generated by 6 766 6 if 6 itself is a afield 39 isaafieldonQandcg C Cg yer am 6A7A cram WW Mime Borel ofield 39 k the kdimensional Euclidean space 1 Q is the real line 6 the collection of all open sets k 76 the Borel afield on 3quot k off 6 is the collection of all closed sets C e k 0 COB B e k is the Borel afield on C Length area volume Length area volume l Definition 12 Let 9454 be a measurable space A set function v defined on 9 is called a measure iff it has the following properties i Og vA oltgtforanyA 69 ii v0 0 iii lfA e 9 i 12 and A s are disjoint ie AMA z for any i7 j then ForanyxEggmxmyxmmifxgt0xooiooifxlt0and Ooo0 o oooltolt o maoltgtforanyagt0 0 mice or oooo is not defined o FOI anyX6MX ltgtXmoltgtlegt0X ltgt7 ltgtlelt0and Ooo0 o oooltolt o maooforanyagt0 o mimor oooo is not defined Probability measure If vQ 1 then v is a probability measure We usually use notation P instead of v Q f is a probability space ifP is a probability measure on a afield on Q Conventions O FOI anyX60 X ltgtX ltgtoltgtlegt0X ltgt7 ltgtlelt0and Ooo0 o oooltolt o maoltgtforanyagt0 o mimor oooo is not defined Probability measure 39 If vQ 1 then v is a probability measure We usually use notation P instead of v Q f is a probability space ifP is a probability measure on a afield on Q Important examples of measures gt 0 Measures take co as its value Important examples of measures 0 Point mass Let X 6 Q be a fixed point c xeA 5XA o X A 0 Counting measure Let Q be a sample space 9 the collection of all subsets and vA the number of elements in A e 9 vA co ifA contains infinitely many elements Then v is a measure on 9 and is called the counting measure 0 Lebesgue measure There is a unique measure m on gig that satisfies mab b 7 a for every finite interval ab foo lt a g b lt co This is called the Lebesgue measure If we restrict m to the measurable space 01 071 then m is a probability measure Proposition 11 Properties of measures Let Q m be a measure space 0 Monotonicity IfA C B then vA g vB 9 Subadditivity For any sequence A1A2 v Agt vA 0 Continuity fA1 CA2 CA3 C orA13A2 3A3 3 and vA1 ltoothen v Jim A Iiim vA where JianAn UA or A 13971 i1 Cumulative distribution function Let P be a probability measure on gig The cumulative distribution function cdf of P is defined to be FX P7oltgtx X 6 Cumulative distribution function Let P be a probability measure on gig The cumulative distribution function cdf of P is defined to be FX P7oltgtx X 6 Proposition 12 Properties of cdf s 1 i Let F be a cdf on Q a Fiw limeimHX 0 b Feo limxsmHX 1 c F is nondecreasing ie FX g Fy ifx gy d F is right continuous ie limy xygtXFy FX ii Suppose a realvalued function F on Q satisfies ad in part i Then F is the cdf of a unique probability measure on gig Stat 709 Mathematical Statistics Lecture 19 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 19 Sufficient statistics and factorization theorem Data reduction without loss of information A statistic TX provides a reduction of the afield 0X Does such a reduction results in any loss of information concerning the unknown population If a statistic TX is fully as informative as the original sample X then statistical analyses can be done using TX that is simpler than X The next concept describes what we mean by fully informative Lecture Stil ireien swam oiomza m film A statistic TX provides a reduction of the afield 0X Does such a reduction results in any loss of information concerning the unknown population If a statistic TX is fully as informative as the original sample X then statistical analyses can be done using TX that is simpler than X The next concept describes what we mean by fully informative J fi ene 391 Let X be a sample from an unknown population P e 9 where 9 is a family of populations A statistic TX is said to be suf cient for P e 9 or for 9 e 9 when 9 P9 9 e e is a parametric family iff the conditional distribution ofX given T is known does not depend on P or 9 151mm M mm 0 Once we observe X and compute a sufficient statistic TX the original data X do not contain any further information concerning the unknown population P since its conditional distribution is unrelated to P and can be discarded o A sufficient statistic TX contains all information about P contained in X and provides a reduction of the data if T is not onetoone O The concept of sufficiency depends on the given family 9 o If T is sufficient for P e 9 then T is also sufficient for P e 90 C 9 but not necessarily sufficient for P e 91 3 9 0 Once we observe X and compute a sufficient statistic TX the original data X do not contain any further information concerning the unknown population P since its conditional distribution is unrelated to P and can be discarded o A sufficient statistic TX contains all information about P contained in X and provides a reduction of the data if T is not onetoone o The concept of sufficiency depends on the given family 9 o If T is sufficient for P e 9 then T is also sufficient for P e 90 C 9 but not necessarily sufficient for P e 91 3 9 r Suppose thatX X1X and X1X are iid from the binomial distribution with the pdf wrt the counting measure f9z 9217 91 ZI0712 z 6 9 6 01 Consider the statistic TX 2 7 X which is the number of ones in X i For any realization X of X X is a sequence of n ones and zeros T contains all information about 9 since 9 is the probability of an occurrence of a one in X and given T t what is left in the data set X is the redundant information about the positions of t ones To show T is sufficient for 9 we compute PX XlT t Let t01nand Bt X1X X O1 2L1Xi t le Bt then PXXT t 0 le 6 Bl then 7 n PXX7T t HPXI Xi 9t1 9quot tHo1xi i1 i1 Also PT t 9 1 e 9n7tl07177nt39 Then PX x7 T t 1 PXXlT t W mlsxx is a known pdf does not depend on 9 Hence TX is sufficient for 9 e O 1 according to Definition 2 4 How to find a sufficient statistic Finding a sufficient statistic by means ofthe definition is not convenient It involves guessing a statistic T that might be sufficient and computing the conditional distribution ofX given T t For families of populations having pdf s a simple way of finding sufficient statistics is to use the factorization theorem How to find a sufficient statistic Finding a sufficient statistic by means ofthe definition is not convenient It involves guessing a statistic T that might be sufficient and computing the conditional distribution ofX given T t For families of populations having pdf s a simple way of finding sufficient statistics is to use the factorization theorem Theorem 22 The factorization theorem 7 Suppose thatX is a sample from P e 9 and 9 is a family of probability measures on 9 29quot dominated by a afinite measure v Then TX is sufficient for P e 9 iff there are nonnegative Borel functions h which does not depend on P on 9 29quot and gp which depends on P on the range of T such that dP Em gpTxhx How to find a sufficient statistic Finding a sufficient statistic by means ofthe definition is not convenient It involves guessing a statistic T that might be sufficient and computing the conditional distribution ofX given T t For families of populations having pdf s a simple way of finding sufficient statistics is to use the factorization theorem quotTheorem 22 The factorization theorem 7 Suppose thatX is a sample from P e 9 and 9 is a family of probability measures on 9 29quot dominated by a afinite measure v Then TX is sufficient for P e 9 iff there are nonnegative Borel functions h which does not depend on P on 9 29quot and gp which depends on P on the range of T such that dP Em gpTxhx To prove Theorem 22 we need the following lemma whose proof can be found in the textbook Vlf aifamily i is dominated by a afinite measure then 9 is dominated by a probability measure Q 21 cP where c s are nonnegative constants with 2101 and P e 9 Vlf aifamily i is dominated by a afinite measure then 9 is dominated by a probability measure Q 21 cP where c s are nonnegative constants with 2101 and P e 9 Emma If a family 9 is dominated by a afinite measure then 9 is dominated by a probability measure Q 21 cP where c s are nonnegative constants with 2101 and P e 9 Proof of Theorem 22 7 i Suppose that T is sufficient for P e 9 For anyA e 9 PAlT does not depend on P Let Q be the probability measure in Lemma 21 By Fubini s theorem and the result in Exercise 35 of 1 6 QA B ichDJAOB icj PAlTde j1 j1 3 4 chAlTdeAPAlTdQ for any B e 0T Hence PAlT EQUAlT as Q where EQUAlT denotes the conditional expectation of IA given T wrt Q Proof of Theorem 22 continued Let gPT be the RadonNikodym derivative dPdQ on the space QTGUL Q Then PA PAiTdP EQIAiTdP EQIAiTgpTdQ dQ EoiIAgpTiT1doIAgpTdo Agpwmdv for any A 6 Hence d x gpmxnmx 1 holds with h dQdv Proof of Theorem 22 continued 1 Let gPT be the RadonNikodym derivative dPdQ on the space 9 76T7 Q Then PA PAlTdP EQIAiTdP EQIAiTgpTdQ EQIAgPTiTidQ IAgpTdo Agpr dv for any A 6 Hence gm gp Txhx 1 holds with h dQdv ii Suppose that 1 holds Then dPidP 7 wiggadvigm ggam 07 2 where the second equality follows from Exercise 35 in 16 Proof of Theorem 22 continued LetA e 0X and P e 9 The sufficiency of T follows from PAlT EQIAlT as P 3 where EQIAlT is given in part i of the proof This is because EQIAlT does not vary with P e 9 and result 3 and Theorem 17 imply that the conditional distribution ofX given T is determined by EQIAlT A e 0X Proof of Theorem 22 continued 1 LetA e 0X and P e 9 The sufficiency of T follows from PAlT EQIAlT as P 3 where EQIAlT is given in part i of the proof This is because EQIAlT does not vary with P e 9 and result 3 and Theorem 17 imply that the conditional distribution ofX given T is determined by EQIAlT A e 0X By 2 dPdQ is a Borel function of T For any B e 0T dP BEQIAlTdPBEQIAlTEdQ BEQ ltA T dQBIAdQBIAdP This proves 3 and completes the proof Exponential famlilies lf 9 is an exponential family then Theorem 22 can be applied with get expln9llt 597 ie T is a sufficient statistic for 9 e e In Example 210 the joint distribution of X is in an exponential family with TX 1X Hence we can conclude that T is sufficient for 9 e O 1 without computing the conditional distribution ofX given T Exponential famlilies lf 9 is an exponential family then Theorem 22 can be applied with 990 eXPn9TT 597 ie T is a sufficient statistic for 9 e e In Example 210 the joint distribution of X is in an exponential family with TXZ 1X Hence we can conclude that T is sufficient for 9 e O 1 without computing the conditional distribution ofX given T Example 211 Truncation families Let x be a positive Borel function on 32 such that ab xdx lt co for any a and b foo lt a lt b lt co Let 9 ab 9 ab 62 a lt b and Mquot C9 XIabxv C9 Example 211 continued Then f9 9 e 9 called a truncation family is a parametric family dominated by the Lebesgue measure on Q Let X1X7 be iid random variables having the pdf f9 Then the joint pdf ofX X17X is I7 Hf9Xi 59nla7wX1I7w7bxn H Xi7 4 H i1 where X is the ith ordered value of X1 X Let X1Xn 99139171392 69nlaywt1Iw7bt2 and hXHl 1 Xi By 4 and Theorem 22 TX is sufficient for 9 e e Example 212 Order statistics Let X X1X and X1X7 be iid random variables having a distribution P e 9 where 9 is the family of distributions on 9 having Lebesgue pdf s Let X X be the order statistics given in Example 29 Note that the joint pdf of X is fx1fXn fX1 fXn Hence TX X1X is sufficient for P e 9 The order statistics can be shown to be sufficient even when 9 is not dominated by any afinite measure but Theorem 22 is not applicable see Exercise 31 in 26 Stat 709 Mathematical Statistics Lecture 21 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 21 Completeness Motivation A statistic VX is ancillary if its distribution does not depend on the population P VX is rstorder ancillary if E VX is independent of P A trivial ancillary statistic is the constant statistic VX E c 6 Q If VX is a nontrivial ancillary statistic then 0VX C 0X is a nontrivial afield that does not contain any information about P Hence if SX is a statistic and VSX is a nontrivial ancillary statistic it indicates that 0SX contains a nontrivial afield that does not contain any information about P and hence the quotdataquot SX may be further reduced A sufficient statistic T appears to be most successful in reducing the data if no nonconstant function of T is ancillary or even firstorder ancillary This leads to the following definition Finding a complete and sufficient statistic Definition 26 Completeness A statistic TX is said to be complete for P e 9 iff for any Borel f EfT O for all P e 9 implies f O as 9 T is said to be boundedy complete iff the previous statement holds for any bounded Borel f Eind ngi and eui iieiem n ersst A statistic TX is said to be complete for P e 9 iff for any Borel f EfT O for all P e 9 implies f O as T is said to be boundedly complete iff the previous statement holds for any bounded Borel f Remarks 0 A complete statistic is boundedly complete 0 If T is complete or boundedly complete and S 11T for a measurable 11 then S is complete or boundedly complete 0 lntuitively a complete and sufficient statistic should be minimal sufficient Exercise 48 0 A minimal sufficient statistic is not necessarily complete for example the minimal sufficient statistic X1X in Example 213 is not complete Exercise 47 Proposition 21 If P is in an exponential family of full rank with pdf s given by fnX eXPnTTXCTIhX7 then TX is complete and sufficient for n e E Proposition 21 v If P is in an exponential family of full rank with pdf s given by 72100 explni x 74nhx7 then TX is complete and sufficient for n e E Proof We have shown that T is sufficient We now show that T is complete Suppose that there is a function f such that EfT O for all n e E By Theorem 21 i ftexpn7t7 71d7l O for all n e E where it is a measure on pP Proof continued 7 Let no be an interior point of E Then ftequotquotdz Ltequotquotdz for all n e N010 1 where N010 n e 3 H71 inoll lt s for some 8 gt O In particular ftequot3tdil Ltequoto dz c If c 0 then f O ae it If c gt 0 then c 1ftequot3 and c 1ftequot3 are pdf s wrt 7L and result 1 implies that their mgf s are the same in a neighborhood of 0 By Theorem 16ii c 1ftequot3 c 1ftequot3 ie f f7 f o ae 7L Hence T is complete Example 215 Suppose that X1 X7 are iid random variables having the Nuc72 distribution u 6 Q 7 gt O From Example 26 the joint pdf of X1 X7 is 2 in29XP711T1712T2 in 7l7 where T1 71Xi T2 72 1XI2 and n 711712 Hence the family of distributions forX X1X is a natural exponential family of full rank E Q x Ooltgt By Proposition 21 TX T1 T2 is complete and sufficient for 71 Since there is a onetoone correspondence between 71 and 9 0102 T is also complete and sufficient for 9 It can be shown that any onetoone measurable function of a complete and sufficient statistic is also complete and sufficient exercise Thus X82 is complete and sufficient for 9 where X and 82 are the sample mean and sample variance respectively Example 216 Let X1X7 be iid random variables from P9 the uniform distribution U 0 9 9 gt O The largest order statistic X is complete and sufficient for 9 e Ooltgt The sufficiency of X follows from the fact that the joint Lebesgue Of X1 7 Xn is Qinlwye From Example 29 X has the Lebesgue pdf nx 19 I079X Let f be a Borel function on 0oo such that EfX O for all 9 gt 0 Then 9 0 fxx 1dx o for all 9 gt 0 Let 69 be the lefthand side of the previous equation Applying the result of differentiation of an integral see eg Royden 1968 53 we obtain that G 9 f99 1 ae m where m is the Lebesgue measure on Ooo 07w Since 69 O for all 9 gt O f99 1 O ae m and hence fx O ae m Therefore X is complete and sufficient for 9 e Ooltgt Example 217 In Example 212 we showed that the order statistics TX X1X of iid random variables X1X7 is sufficient for P e 9 where 9 is the family of distributions on 9 having Lebesgue pdf s We now show that TX is also complete for P e 9 Let 90 be the family of Lebesgue pdf s of the form fx C91 9exp7X2 91X 92X2 9X where 9 e 9 and C91 97 is a normalizing constant such that ffxdx 1 Then 90 C 9 and 90 is an exponential family of full rank Note that the joint distribution ofX X1 X is also in an exponential family of full rank Thus by Proposition 21 U U1 U is a complete statistic for P e 90 where U 2771 Xf Since as 90 implies as 9 UX is also complete for P e 9 Examle 217 continued The result follows if we can show that there is a onetoone correspondence between TX and UX Let V1 ZF1XI39 V2 Ziq XI va V3 ZiltjltkXinkauy Vn X1quot Xn From the identities Uk V1 Uk71 V2Uk72 1k71Vk71U1 1kak 07 k 1 n there is a onetoone correspondence between UX and VX V177vn From the identity t7X1t7X t LV1t H V2t 2771 v there is a onetoone correspondence between VX and TX This completes the proof and hence TX is sufficient and complete for P e 9 In fact both UX and VX are sufficient and complete for P e 9 The relationship between an ancillary statistic and a complete and sufficient statistic is characterized in the following result Theorem 24 Basu s theorem Let V and T be two statistics ofX from a population P e 9 If V is ancillary and T is boundedly complete and sufficient for P e 9 then V and T are independent wrt any P e 9 The relationship between an ancillary statistic and a complete and sufficient statistic is characterized in the following result Theorem 24 Basu s theorem Let V and T be two statistics ofX from a population P e 9 If V is ancillary and T is boundedly complete and sufficient for P e 9 then V and T are independent wrt any P e 9 Let B be an event on the range of V Since V is ancillary PV 1B is a constant As T is sufficient EIBVlT is a function of T not dependent on P Because EEBVlT7PV 1B o for all P e 9 by the bounded completeness of T PV 1BlT EIBVlT PV 1B as 9 Proof continued Let A be an event on the range of T Then PT 1A V43EEIATIBVTEIATEIBVT EIATPV 1B PT 1APV 1B Hence T and V are independent wrt any P e 9 Proof continued Let A be an event on the range of T Then PT 1A V4 5 EEATBViT EATEBViT EIATPV 1B PT 1APV 1B Hence T and V are independent wrt any P e 9 Basu s theorem is useful in proving the independence of two statistics Proof continued Let A be an event on the range of T Then PT 1A V4 5 EEATBVlT EATEBVlT EIATPV 1B PT 1APV 1B Hence T and V are independent wrt any P e 9 l Rem enrlk i Basu s theorem is useful in proving the independence of two statistics J Example 218 Suppose that X1 X7 are iid random variables having the Nuc72 distribution with u 6 Q and a known 7 gt 0 It can be easily shown that the family Nuc72 u e 3 is an exponential family of full rank with natural parameter n ucr2 By Proposition 21 the sample mean X is complete and sufficient for n and u Example 218 continued Let 82 be the sample variance ince 82 n71 12 1Z722 where Z X in is NO7c72 and Z n 1zl 71Z 82 is an ancillary statistic 72 is known By Basu s theorem X and 82 are independent wrt Nuc72 with p 6 Since 72 is arbitrary X and 82 are independent wrt Nuc72 for any 11 6 Q and 72 gt O i Let 82 be the sample variance Since 82 n71 12 1Z722 where Z X in is NOc72 and Z n 1zl 71Z 82 is an ancillary statistic 72 is known By Basu s theorem X and 82 are independent wrt Nuc72 with u 6 Since 72 is arbitrary X and 82 are independent wrt Nuc72 for any 11 6 Q and 72 gt 0 Using the independence of and 82 we now show that n7132c72 has the chisquare distribution 95371 Note that 2 2 if n71 32 n X7 nlt gt 2lt39 gt o o 1 a From the properties of the normal distributions n7u2c72 has the chisquare distribution 9512 with the mgf 1 72042 and 2 1X 7 u2c72 has the chisquare distribution 95 with the mgf 1720472 tlt1 2 Example 218 continued By the independence of and 82 the mgf of n7132c72 is 1 72tquot2172t12 172tquot12 for t lt 12 This is the mgf of the chisquare distribution 952771 and therefore the result follows Stat 709 Mathematical Statistics Lecture 39 Jun Shao Department of Statistics University of Wisdonsin Madison WI 53706 USA Lecture 39 The method of moments An exactly unbiased estimator may not exist or is hard to obtained We often derive asymptotically unbiased estimators The method of moments is the oldest method of deriving asymptotically unbiased estimators although they may not be the best estimators Consider a parametric problem where X1X7 are iid random variables from P9 9 e e C k and ElX1lk lt co Let ul EX4 be the jth moment of P and let be the jth sample moment which is an unbiased estimator of u j 1k Method of moments Typically MMWL jLmK n for some functions h on k By substituting uj s on the lefthand side of 1 by the sample moments we obtain a moment estimator ie satisfies ML JLwh which is a sample analogue of 1 This method of deriving estimators is called the method of moments An important statistical principle the substitution principle is applied in this method Method of moments Typically ujhj9 j1k 1 for some functions h on k By substituting uj s on the lefthand side of 1 by the sample moments we obtain a moment estimator ie satisfies hj 7 11k which is a sample analogue of 1 This method of deriving estimators is called the method of moments An important statistical principle the substitution principle is applied in this method Let fl 1 1v k and h 717hk Then h9 lfthe inverse function h 1 exists then the unique moment estimator of 9 is 97h 1u Method of moments When 1 1 does not exist ie h is not onetoone any solution of f1 h9 is a moment estimator of 9 A If possible we always choose a solution 9 in the parameter space 9 In some cases however a moment estimator does not exist see Exercise 111 NHMMHMEMIIIIIIIIIIIIIIII When h 1 does not exist ie h is not onetoone any solution of f1 h is a moment estimator of 9 If possible we always choose a solution 3 in the parameter space 9 In some cases however a moment estimator does not exist see Exercise 111 Assume that 3 90 for a function g If h 1 exists then 9 h 1 If g is continuous at u 1uk then 3 is strongly consistent for 9 since j Ha W by the SLLN If g is differentiable at u and ElX1l2k lt co then is asymptotically normal by the CLT and Theorem 112 and mwm wmwwwm where V is a k x k matrix whose ijth element is MH 7 My Furthermore the n 1 order asymptotic bias of is QWWWMmm Example 324 Let X1X7 be iid from a population P9 indexed by the parameter 9 0102 where u EX1 6 Q and 72 VarX1 e Ooltgt This includes cases such as the family of normal distributions double exponential distributions or logistic distributions Table 12 page 20 Since EX1 u and EX12 VarX1 EX12 72 u2 setting m u and g 72 u2 we obtain the moment estimator 3 lt2 1x7gt39lt2gt 2 32 Note that X is unbiased but 82 is not

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "I made $350 in just two days after posting my first study guide."

#### "There's no way I would have passed my Organic Chemistry class this semester without the notes and study guides I got from StudySoup."

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.