MATHEMATICAL STATISTICS I
MATHEMATICAL STATISTICS I M 384C
Popular in Course
Popular in Mathematics (M)
This 4 page Class Notes was uploaded by Reyes Glover on Sunday September 6, 2015. The Class Notes belongs to M 384C at University of Texas at Austin taught by Staff in Fall. Since its upload, it has received 48 views. For similar materials see /class/181478/m-384c-university-of-texas-at-austin in Mathematics (M) at University of Texas at Austin.
Reviews for MATHEMATICAL STATISTICS I
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 09/06/15
STATISTICAL PROPERTIES OF LEAST SQUARES ESTIMATORS Recall Assumption EYx no nlx linear conditional mean function Data X1y1 X2 yz xn yquot Least squares estimator 1le 1 f71x where A SXY A A 771 7103 quot77135 SXXZEXF 2EXiXi39E SXYEXi7Yi EXi7Ya Comments 1 So far we haven t used any assumptions about conditional variance 2 If our data were the entire population we could also use the same least squares procedure to fit an approximate line to the conditional sample means 3 Or if we just had data we could fit a line to the data but nothing could be inferred beyond the data 4 Assuming again that we have a simple random sample from the population If we also assume ex equivalently le is normal with constant variance then the least squares estimates are the same as the maximum likelihood estimates of no and n1 Properties of n and 1 x 7 ASXY y xi c 1771 y 20iyi SXX SXX H SXX H Xi f where ci SXX Thus If the xi s are fixed as in the blood lactic acid example then l is a linear combination of the yi39s Note Here we want to think of each yi as a random variable with distribution lei Thus if the yi s are independent and each lei is normal then l is also normal If the lei39s are not normal but n is large then l is approximately normal This will allow us to do inference on 1 Details later 1 2 E ci E EX Xx xi 7 0 as seen in establishing the alternate expression for SXX x T 1 SXX 1 SXX SXXEX OC x SXX 3ExiciEXi Remark Recall the somewhat analogous properties for the residuals a A A 1 1 4 170 y 471x 2 E y Eelin E ci yi also a linear combination ofthe yi s i1 i1 i1 hence 5 The sum of the coefficients in 4 is l c f l 176 nl 170 1 i1 n i i1 n i1 i n I Sampling distributions of n and 1 Consider x1 xn as fixed ie condition on x1 x Model Assumptions quotThequot Simple Linear Regression Model Version 3 39 EYx no nlx linear conditional mean function 39 VarYx 02 Equivalently Varex 02 constant variance 39 NEW yl yn are independent observations independence The new assumption means we can consider yl yn as coming from n independent random variables Y1 Yquot where Yi has the distribution of lei Comment We do not assume that the xi s are distinct If for example x1 x2 then we are assuming that y1 and y2 are independent observations from the same conditional distribution lel Since Y1 Yn are random variables so is f71 but it depends on the choice of XI xquot so we can talk about the conditional distribution llxl X n Expected value of 71 as the yi39s vary Ef71x1 XnEEciYilx1 xn 71 2 Sci FYix1 xquot 2 Sci EYilxi since Yi depends only on x 20 710 nlxi model assumption 2 quot020i 71120 Xi 7100 7111 711 Thus l is an unbiased estimator of 111 Variance of l as the yi39s vary Varf71x1 xquot VarECiYix1 xquot i1 2 Eci2 VarYix1 xquot Eci2 VarYixi since yi depends only on x Ecizoz 0220 x 7 2 022 SXX definition of ci 02 2 SXX2 20C x 02 SXX 02 For short Var A 771 SXX sd l 2 Comments This is vaguely analogous to the sampling standard deviation for a mean y population standard deviation sd estimator lsomething However here the quotsomethingquot namely SXX is more complicated But we can still analyze this formula to see how the standard deviation varies with the conditions of sampling For y the denominator is the square root of n so we see that as n becomes larger the sampling standard deviation of y gets smaller Here recalling that SXX E xi 4 2 we reason that 39 If the xi s are far from 27ie spread out SXX is so sd 1 is 39 If the xi s are close to fie close together SXX is so sd 1 is Thus if you are designing an experiment choosing the xi s to be from their mean will result in a more precise estimate of 1 Assuming all the model conditions fit Expected value and variance of 1 A 1 Using the formula no 2 Ci yi calculations left to the interested student s1m11ar i1 n to those for l will show 39 E670 no So no is an unbiased estimator of no 1 r2 39 Var A 02 so 17quot n SXX A 1 r2 d 0 S 17quot n SXX Analyzing the variance formula 39 A larger 7 gives a variance for 1 gt Does this agree with intuition 39 A larger sample size tends to give a variance for 1 39 The variance of n is except when E lt 1 than the variance of 11 gt Does this agree with intuition 39 The spread of the xi s affects the variance of 1 in the same way it affects the variance of 1 Covariance of n and 1 Similar calculations left to the interested student will show TC Cov A A c72 no 771 SXX Thus 39 n and l are not independent except possibly when gt Does this agree with intuition 39 The sign of Covf70f71 is opposite that of f gt Does this agree with intuition Estimating 02 To use the variance formulas above for inference we need to estimate 02 2 VarYX the same for all i First some plausible reasoning If we had lots of observations yil y yim from lei then we could use the univariate standard deviation 1 2 m 1 20quot Yr of these m observations to estimate 02 Here i is the mean of yil y yim which would be our best estimate of EYI xi just using yil y yim We don39t typically have lots of y s from one xi so we might try reasoning that EY xi is our best estimate of EYIXQ 1 quot A 2 Eyi EYxi quot 1 i1
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'