Introduction to Mathematical Statistics
Introduction to Mathematical Statistics STAT 312
Popular in Course
Popular in Statistics
Mrs. Triston Collier
verified elite notetaker
This 31 page Class Notes was uploaded by Mrs. Triston Collier on Thursday September 17, 2015. The Class Notes belongs to STAT 312 at University of Wisconsin - Madison taught by Staff in Fall. Since its upload, it has received 18 views. For similar materials see /class/205088/stat-312-university-of-wisconsin-madison in Statistics at University of Wisconsin - Madison.
Reviews for Introduction to Mathematical Statistics
Report this Material
What is Karma?
Karma is the currency of StudySoup.
Date Created: 09/17/15
Stat 312 Lecture 24 Chi square Goodness of t Test Moo K Chung mchung statwiscedu April 24 2003 Concepts 1 The null hypothesis of interest is H0 2131 01wpk 01 H1 213339 7 Cj for some 9 2 The test statistic 2 N1 7 EN12 Nc 7 ENk2 EN1 39 ENc Assuming EN7 2 5 for every 9 X2 N Xiil39 Proof When k 2 binomial data it is easy to prove Note that X2 We have shown that for large n Nln PI P11P1n X2 goodness of t test can be used to test whether sample data follows a particular dis tribution N N01 03 Inclass problems Example 1 There are four blood types ABO sys tem It is believed that 34 15 23 and 28 of peo ple have blood type A B AB and 0 respectively The blood samples of 100 students were collected A12 B56 AB2 030 Test if the collected blood sample contradicts the previous belief Solu tion The null hypothesis would be H0 2 13A 034133 0153AB 023130 028 gt chi lt 1234quot2345615quot215 223quot2233028quot228 1 1456187 gt 1pchisqchi3 1 0 Since the P value is almost 0 we reject H0 Example 2 90 rats proceed down a ramp to one of three doors Test if the rats have no pref erence concerning the choice of a door when the observed frequencies for the three doors are m 23712 36713 31 Solution We compute X2 287 The P value the goodness of t test is 1 7 pchisq2872 024 Since this is somewhat large we do not reject H0 Example 3 The number of accidents X per week at an intersection was checked for 50 weeks 32weeks with no accident 12 weeks with 1 accident 6 weeks with 2 accidents Test if X follows a Pois sion distribution Solution H0 PX w Aze Awl for all x 012 Since A unknown we estimate it with MLE X 2 12 2 X 650 048 De ne three categories by X 0X 1 and X 2 2 The expected probabilities are estimated to give 131 0619132 0297133 17131 7132 0084 Then the test statistic X2 N x the additional de gree of freedom is lost due to the estimation of A X2 1354 and the P value would be 025 We do not reject H0 Selfstudy problems Example 142 148 149 Stat 312 Lecture 9 Chi squared distributions Moo Chung mhungstatwiscodu August 26 2002 Concepts 1 Let X1MXn be a random sample from N 1102 wt 1 1 7 2 2 i Xi X 27 02 0211 N Critical values for xiii PltXiziu2n lt lt xix2 1 a 3 1001 a CI for 02 1 1 quot782 lt 2 lt Vi gt82 Xuznil Xliaznil 3 X exp exponential distribution with pa rameter The density function is de ned for x 2 itiiiCA f it In class problems Continuing Example 712 Fat content of 10 ran domly selected hot dogs Assuming that observa tions come from normal distribution nd a 95 CI for the population variance of fat content gt xltc252213228170298210 255160209195 gt Sdx 1 4 13414 gt qchisq09759 1 1902277 gt qchisq00259 1 2700389 gt 9qchisq00259sdx 1 1377848 gt 9qchisq09759413414 1 1955933 Review problem Lifetime of an electrical component has an expo nential distribution with parameter Estimate by matching moments gt xltc415318732993034123311752 7302223634002678 gt meanx 1 55087 Obtain the maximum likelihood estimate of the probability that lifetime exceeds 10 gt exp105509 1 08340005 Suppose that there are 100 electrical components with 5 55 40 Find a 95 con dence interval for A gt 55qnorm0 975 4010 1 8283986 gt 55qnorm0 975 4010 1 47 18014 Self study problems Example 715 Exercise 743 N L Stat 312 Lecture 04 Moment Matching Moo K Chung mchungstatwisc edu September 14 2004 Given arandom sample X17 7X a linear estimator of parameter 6 is an estimator of form l1 Then it can be shown that X is the MVUE for population mean among all linear unbi ased estimators Proof Case n 2 will be proved The gen eral statement follows inductively Consider linear estimators Cle Cng To be unbiased 01 02 1 To be most ef cient among all unbiased linear estimators the variance has to be minimized The vari ance is viz cfvx1 cgvxz c 1 i 502w The quadratic term in the bracket 20 7 201 1 is minimized when 01 12 Given a random sample X17 7X the k th sample moment is Mk 21 The moment estimators of population param eters are obtained by matching the sample moments to correspond population moments and solving the resulting equations simulta neously Exponential distribution X is an exponen tial distribution with parameter A if the den sity function is fx Ae for z 2 0 It can be shown thatX VX 4 Given random sample X17 7X the like lihood function is given as the product of probability or density functions ie L09 f9619f9629 f0 The maximum likelihood estimatate oft9 is an estimate that maximizes L09 If we denote 0x1 7x to be the maximum like lihood estimate The maximum likelihood estimator MLE of 6 is denoted by d x1 Xn Review Problems Example 612 Example 616 Example 617 l N E Stat 312 Lecture 03 Minimum Variance Unbiased Estimator M00 K Chung mchungstatwiscedu September 9 2004 Population of interest is a collection of measur able objects we are studying Let X17 7Xn be a random sample from the population Then sam ple mean X and sample variance 52 are unbiased estimators of population mean u and population variance 02 respectively Proof Note that 1 e 1 n 2 Earn n 71Xi27nX2 Then using the fact EX2 VX E502 0271 p2 it can be shown that E52 a There may be many unbiased estimators of 0 Given two unbiased estimators 01 and 02 of 0 We choose one that gives less variance If 370 V 2 l is called more ef cient than z An ef cient estimator has less variability so we are more likely to make an estimate close to the true param eter value The following coin ipping example clearly demonstrate this gt alt rbinom1000l05 gt a lOOlOOlOllOO gt meana 1 0517 gt meanal3 1 03333333 gt meanal9 1 04444444 gt meanalll 1 03636364 Among all unbiased estimators we choose the most ef cient estimator called the minimum vari ance unbiased estimator MVUE The MVUE is an unbiased estimator with the smallest variance MVUE is the most ef cient estimator An ef cient estimator will produce an estimate closer to the true parameter 0 X is MVUE for u we will not prove this statement 4 Given a random sample X17 7Xn a linear es timator of parameter 9 is an estimator of form 71 Z i1 Then it can be shown that X is the MVUE for population mean u among all possible linear esti mators Proof Case n 2 will be proved The general statement follows inductively Consider linear es timators Ill 3le Cng To be unbiased 01 02 1 To be most ef cient among all unbiased linear estimators the variance has to be minimized The variance is v 2le 3ng 0 17 cl az The quadratic term in the bracket 20 7 201 l is minimized when 01 12 Review Problems You are not required to do these problems but these are problems you should be able to answer after each lecture What is an unbiased estima tor of population parameter uz Exercise 63 Stat 312 Lecture 21 Linear Regression 11 M00 K Chung mchungstatwisc edu Dec 7 2004 Concepts 1 Equot Maximum likelihood estimation Given a linear model Yj o lmj5j with e N N07 02 we will estimate 3031 using the maximum likelihood estimation Note that Yj N NWO 51 02 The density function for are I 7 1 24739 i 30 7 Bjmjy ya 7 72m exp 202 1 The loglikelihood is 30731 const 7 T j 24739 r 50 519012 H H We maximize the loglikelihood which is equivalent to minimizing the sum of the residuals 1 7L g ZQJJ39 50 519012 j1 Hence MLE is LSE in linear regression The least squares estimation for 31 are given by A 7 SM 51 7 Sm where Sm n 7 g It can be shown that A S n 51 2 0M7 j1 where Cj j 7 So LSE is a linear estimation From 21 07 0 2991 cjzj 1 and 2991 c 5 we can show that 11351 51 showing unbiasness Further V31 LizSm Since 31 is a linear combination of normals 31 N NBlv 02sz We are interested in hypothesis testing H02310VSH12317 0 W Inference on the slope parameter 31 is based on test statistic A T N 757L727 r where SE1 xSm and 7 SSEn 7 2 It can be shown that SSE SW 7 llSm we get S1 m 7 Then we reject H0 if 1251 gt twin at 1001 7 00 signi cance We don t usually compute the test statistic by hand Use Rpackage Example We continue Lecture 19 example gtsummarylmyx Call lmformula y x Residuals Min 1Q Median 3Q Max 10908 6312 1758 4354 10836 Coefficients Estimate Std Error t value Prgtt Intercept 2948 1323 222 006 X 055 017 312 001 Signif codes 0 0001 001 Residual standard error 7647 on 8 degrees of freedom Multiple R Squared 05519 Adjusted R squared04959 F statistic9854lt l and 8 DE p value 001383 Stat 312 Lecture 17 Simple Linear Model M00 K Chung mchungstatwiscedu March 27 2003 Concepts 1 Deterministic model g 7 a y 50 5196 g 39 For example as is the speed of a car and y is the distance the car traveled in 1 hour This is an unrealistic model a 7 B because the car can not maintain the absolute constant gt speed So we introduce a noise term 6 in the above equa tion E 5 2 Stochastic model let 6 be a random noise with Es 0 j 39 an Vare 02 Instead of the deterministic model we a will have y o 1xe Since 6 is a random variable we use Y instead of y Y 051966 Note thatlEY o lm and VarY 02 gt XltC 8039 7539 6039 9039 9939 6039 5539 85 65 70 3 Givenpaired data 951 yl yon we have a lin gt ylt c 7O 6O 70 72 95 66 6O ear stochastic relationship 8 O 7 O 60 gt plot X y Yj 0 1xjej gt rarara lt lmyx where yj is the observed value of a random variable gt rarara and Ej N 6 Note that o lxj Let 3031 Call 2 I 1r formuj39a y N X be estimators of g l Then the predicted values or COEfflClentS Intercept tted values j o lej are estimators of I 2 9 4 827 0 5523 g lxj The differences between the observations gt abllne rarara yj and the predicted values j are called the residuals I errors Le gt llbrary DevoreS A A gt data 6X12 15 T 7 y 7 7 y 7 o 7 lxj gt attach ex12 15 gt ele 15 4 Least squares estimation We estimate g and l by Mo strength minimizing the sum afthe squarederrars SSE 1 2 9 g n n n 2 33 2 7 2 SSE ZWrEV Z r or lxj j1 j1 j1 gt lm StrengthNMoE Then the regression line is y o lm gzii inuj39a y X Intercept X In class problems 3 2 9 2 5 0 10 7 5 Example I 10 students took two midterm exams Student 0102 03 04 05 06 07 08 0910 Midterm 1H 80 75 60 90 99 60 55 85 65 70 self Study pmblems Midterm 2H 70 60 70 72 95 66 60 80 70 60 Let s nd the regression line Example 123 124 126 127 128 1 The P value is the smallest level of signi cance to Stat 312 Lecture 15 Ij vahies Moo K Chung mchung statwiscedu November 27 2004 less than 100 95 percent confidence interval Inf 788531 sample estimates mean of x 625 at which H0 would be rejected Example 857 815 Nicotine contents X17 X32 of cigarettes follow Np022 We are interested in testing if 03 P value computation for two sided tests Let us continue the nicotine content example and H0 u 15 vs H1 11 gt 15 nd the P value for testing Assume i 157 Given Oz level7 the corre sponding rejection regions are given by H0u15vs H1u7 15 gt 21pnorm1979899 gt qnorm005 1 004771488 1 1644854 gt qnorm0025 1 1959964 gt qnorm001 1 2326348 gt qnorm0005 1 2575829 Review Problems Example 8177 818 gt 157 1502sqrt32 1 1979899 gt 1pnorm1 979899 1 0 02385744 P value a a reject H0 at level a P value gt 04 a do not reject H0 at level a The smaller the P value7 the easier to reject H0 Example Dog IQ example gt ttestxmu100alternativequotlessquot conflevel095 One Sample ttest data x t 42036 df 9 pvalue 0001147 alternative hypothesis true mean is Stat 312 Lecture 15 Two sample t test Moo K Chung mchung statwiscedu March 13 2003 Concepts 1 Pooled sample variance 3201 71gts g m e vs p n m 7 2 2 Let X1 Xn and Y1 Ym be two indepen dent samples from normal distributions with the same population variance The test statis tic for testing HOIMXWV8H1IMX7W XeYeinmd prln1m Reject H0 if T gt taZ nm2 N tnm72 r In class problems Example 1 A study was conducted to compare the weights of cats and dogs Weights of cats 20 21 35 13 21 10 Weights of dogs 31 10 20 40 Assume that the population variance to be same for both cats and dogs Is there any difference between the weights of cats and dogs gt xltc202135132110 gt ylt c31102040 gt sqrt5varx3vary8 1 1052824 gt tmeanxmeany1053sqrt1513 gt t 1 06827026 gt qt0058 1 1859548 If you use R it is very easy to do two sample hy pothesis testing gtttestxyalternativequottwosidedquot varequalTRUEconflevel09 Two Sample ttest datazx and y t 07725 df 8 0462 alternative hypothesis true difference in means pvalue is not equal to confidence interval 1788739 738739 gtttestxyalternativequottwosidedquot conflevel09 Welch Two Sample ttest datazx and y t 07073df 4778 05124 alternative hypothesis true difference in means pvalue is not equal to confidence interval 20361647 9861647 Example 2 A cat named Tom catches average 3 mice per day with the sample standard deviation of 1 for 10 days A cat named Jerry catches average 4 mice per day with the sample standard deviation of 12 for 8 days Which cat do you think mice would hate the most at level 005 gt sltsqrt91quot2712quot21082 gt 34ssqrt11018 1 1930543 gt qt00516 1 1745884 Selfstudy problems Compute a CI for uX 7 uy in Example 1 Please read p 370 section Pooled t Procures Example 96 97 using Concept 2 Homework 5 Due April 1 1100am Exercise 850 854 92 94 920 922 Use Concept 2 when you do two sided t tests Stat 312 Lecture 10 Hypothesis testing Moo K Chung mchung statwiscedu Feb 20 2003 Concepts gt meanx 1 625 1 The null hypothesis H0 is a claim about the value of a population parameter The alternate hypothesis H1 is a claim opposite to H0 I A test of hypothesis is a method for using sam ple data to decide whether to reject H0 H0 will be assumed to be true until the sample evidence suggest otherwise 03 A test statistic is a function of the sample data on which the decision is to be based A rejection region is the set of all values of a test statistic for which H0 is rejected 01 Type I error you reject H0 when H0 is true PType I error Preject HOIHO true Oi The resulting Oi is called the signi cance level of the test and the corresponding test is called a level Oi test We will use test procedures that give Oi less than a speci ed level 005 or 001 Inclass problem I believe that dogs are as smart as people Assume IQ of a dog follows Xi N Na102 IQ of 10 dogs are measured 30 25 70 110 40 80 50 60 100 60 We want to test if dogs are as smart as people by testing HQZIU100VS H1alt100 One reasonable thing one may try is to see how high the sample mean is gt xltc30 25 70 110 40 80 50 60 100 60 Since the average IQ of 10 dogs are lower than 100 one would be inclined to reject H0 Let X be a test statistic and R 70090 to be a rejection region Let s compute the probabil ity of making Type I error based on this testing procedure Under the assumption H0 is true X2 N N100102 Under this condition X N N100 10 and a PX g 90 gt pnorm90 100sqrt10 1 00007827011 By using this test procedure it is highly unlikely to make Type I error Let s see what happens when we change the rejection region When R 70095 Oi PX S 95 gt pnorm95 100sqrt10 1 005692315 When R 70099 a PX g 99 gt pnorm99 100sqrt10 1 03759148 The test procedure based on rejecting H0 if X S 99 will produce huge Type I error Why Selfstudy problems Example 81 82 83 84 85 Do not compute 8 1 E 5 Stat 312 Lecture 02 Point Estimation M00 K Chung mchungstatwiscedu September 2 2004 IfX and Y are independent 1EX Y EX 1 Taking expectation on both sides we de ne the EY and VX Y VX WY See pp 244 bias of estimator 9 to be If a random sample Xi N NL70392 X N Biaso 7 939 No 0271 Proof It measures the biasness of estimator 9 in average 7 I 2 I 2 sense VX Him W101 71 1 n Reviw Problems example 6163 X02 7 2 Assignment 1 Due Sept 16 930am Exercise 62 68 610 616 620 622 Now note that m X2 w Xi 2th 2m DEW M w Hence VX A point estimate of population parameter 9 is a single number that reasonably approximate 0 A point estimator of 9 is a statistic rule or formula for getting the point estimate given sample data Let Xi be a Bernoulli random variable with PXl 1 p Then 21Xi is a Binomial random variable with n and p parameters Note that a Binomial random variable with n 1 is a Bernoulli random variable The following exam ple simulates 1000 coin tosses gt alt rbinom1000l05 gt a l00100101100 gt suma1000 1 0517 gt blt rbinom1000000l05 gt sumb1000000 1 0500545 We can let 9 error of estimation Stat 312 Lecture 14 Two sample tests Moo K Chung mchung statwiscedu March 11 2003 Concepts 1 Let X1Xn and Y1Ym be two in dependent samples with mean MX My and variance 0 0 respectively EOZ Y MX MY VarX 7 Y 0 n Tam 2 Let X1Xn and Y1 Ym be two inde pendent samples with X N NmX 0 and Y N Nuy032 Then X 7 Y Z N N0 1 Ma n 0 m 3 Assume n and m to be large The test statistic for testing HOIMXMYVS H13HX7 MY ZLNN07U x03cn0m In class problems Example 1 A study was conducted to compare the reaction times of men and women to a stim ulus 12 men and 9 women were employed in the experiment Let X be the reaction time of the i th man and be the reaction time of the j th woman There is a reason to believe that X N NMX018 and Y N NMy014 We want to test if the population means are the same The following measurements were taken i 36 seconds y 38 seconds ls there any di erence between the reaction times of men and women gt z3638Sqrt01812O149 gt Z 1 1144155 Example 2 A study was conducted to compare the reaction times of men and women to a stim ulus 50 men and 65 women were employed in the experiment Let X be the reaction time of the i th man and be the reaction time of the j th woman The following measurements were taken i 36 seconds y 38 seconds 3 018 3 014 ls there any di erence between the reaction times of men and women gt z3 6 3 8sqrt0185001465 gt z 1 263664 gt qnorm0025 1 1959964 What is the main reason you are getting the di erent results in Example 1 and 2 Selfstudy problems Determine the P Value in Example 1 and 2 Stat 312 Fall 2003 11132003 Discussion 10 1 The Analysis of Variance An ANOVA Table Source of Variation df Sum of Squares Mean Squares 1 Treatment I 71 SSTr MSTrSSTrI 71 MSTrMSE Error IJ 71 SSE MSESSEIJ 71 Total IJ 7 1 SST The total sum of squares SST treatment sum of squares SSTr and error sum of squares SSE are given by I J I 1 SST Zszj7 z2 111 11 11 ssn 7 ii 11271i112 1112 7 l 7 11 71 J 11 1 I J J I J SSE 7 ii2 Where 1 Zij m 11 11 71 11 11 11 Example 1 Exercise 6 page 421 Devore 6th The article 7 Origin of Precambrian lron Formations Econ Geology 1964 1025 1057 reports the following data on total Fe for four types of iron formation 1carbonate 2silicate 3mag netite 4hematite 205 281 278 270 280 252 253 271 205 313 263 240 262 202 237 340 171 268 237 249 295 340 275 294 279 262 299 295 300 356 365 442 341 303 314 331 341 329 363 255 Carry out an analysis of variance F test at signi cance level 01 and summarize the results in an ANOVA table 12 Example 2 Exercise 8 page 421 Devore 6th A study of the properties of metal plateconnected trusses used for roof support 77 Modeling Joints Made with Light Gauge Metal Connector Plates77 Forest Products J 1979 39 44 yielded the following observations on axial stiffness index kipsin for plate lengths 4 6 8 10 and 12 in 4 3092 4095 3110 3265 3168 3498 3097 6 4021 3472 3610 4045 3310 3489 3817 8 3924 3662 3510 3571 4099 3673 3820 0 3467 4529 4614 4331 4106 3842 3626 2 4074 4418 4199 4107 4734 4412 4658 HH Does variation in plate length have any effect on true average axial stiffness State and test the relevant hypotheses using analysis of variance with 01 01 Display your results in an ANOVA table Hint 229527 5 24142079 TingLi Lin Stat 312 Fall 2003 11132003 2 Multiple Comparison in ANOVA w QaIIJ71 MSEJ 21 Example 3 Exercise 14 page 427 Devore 6th Use Tukey s procedure on the data in Exercise 4 to identify differences in true average ight times among the four types of mosquitos Recall i1 4397 922 4527 923 5497 924 6367 92 5197 and 229 91191 22 Example 4 Exercise 18 page 428 Devore 6th Consider the accompanying data on plant growth after the application of different types of growth hormone 1 13 17 7 14 2 21 13 20 17 Hormone 5 18 15 20 17 4 7 11 18 10 5 6 11 15 8 a Perform an F test at level 04 05 b What happens when Tukey s procedure is applied TingLi Lin Stat 312 Lecture 24 Goodness of t Test for Distributions Moo K Chung mchung statwiscedu April 29 2003 Concepts 1 Knowing the exact distribution of a sample is important for statistical analysis We want to test if a sample 1 6 wn follows a certain probability distribution P We perform a X2 goodness of t test on the null hypothesis H0PX C pi for alli 1 k For a continuous distribution 0 are intervals In class problems Example 1 Let us see if the distribution of output tuft weight Example 1411 follows an exponential distribution with parameter 1 l Interval l 0 8 l 8 16 l 16 24 l 24 32 l 32 40 l 40 48 l 48 56 l 56 64 l 64 70 l l0bservedl20l8l7l1l2l1lolllOl Solution The density of the exponential random variable X is given by fw e z w 2 0 Note that 08 e z i 05 Solving this we estimate 31 1154 Based on this we compute the expected number of elements in each category For instance the expected number of element in the interval 8 16 is 40 08 ax1154 dw 40pexp16 11154 7 pexp811154 gt intervalltc81624324048566470 gt plt pexpinterval1I 1 050 075 087 094 097 098 099 0996 0998 gt p29p18 1 025 012 006 003 0016 0008 0004 0002 gt 40p29p18 10 5 25 13 06 03 02 006 l Expected l 20 l 10 l 5 l 25 l 13 l 06 l 03 l 02 006 l Then we compute X1 50 and the P value is pchisq507 0 So we reject H0 and conclude that the exponential distribution is not a good t Selfstudy problems Example 1411 This is slightly different from the in class example setting Exercise 1214 Assignment 8 Due May 8 1100am 146 1416 1418 1426 1432 Stat 312 Lecture 19 Least squares estimation M00 K Chung mchungstatwisc edu Nov 30 2004 Concepts 1 Least squares method In the previous lecture we stud ied the least squares estimation method for estimating parameters in linear regression This method can be used to estirn ate other parameters in a different model Given measurements yl yg yn they can be modeled as Yr M 51397 where lEei 0 and Vei a2 with no assumption of normality We are interested in estimating u lEY the population mean Let t be the estimator of M Then the predicted value is i t and the residual error is n yi 7 Q So the total sum of the squared errors SSE is SSE 2o 7 a i1 To nd minimum of SSE we differentiate SSE with re spect to t and get t g N Weighted least squares method Suppose we have two population Measurement are taken from the rst pop ulation 1112 rm and the second population yr y2 yn They are modeled as Xi M 6i with lEei 0 and Vei 01 while Yz M 6i with lEei 0 and Vel 0392 Errors ei and ej are assum ed to be independent We can again use the least squares method to estimate the common popula tion mean u but this problem is more complicated than the previous case due to the different variabilities We solve this problem by transforming the above equations to have the same variabilities X i 61 i 7 01 01 01 E M 61 0392 Then 571 and 1 12 are iid and SSE is given by U 1 m A 1 7 A SSE g W2 g r it 1 i1 2 i1 We again differentiate SSE with respect to ft The non trivial solution is 1 m 7 n 7 Aggggggggi 471 47 mar nas ar y This method is called the weighted least squares method Example Line probe measurements of mice micro CT breast cancer imaging Download data les from httpwwwstatwiscedumchung teachingdatalineprobe into your computer di rectory gt alt readtablequotdzstat312probeltxt quotheaderT a gt probel l 4033O 2 37964 3 35926 gtprobel lt aprobel gt probel l 4033O 37964 gt meanprobel 1 3898323 gt var probel 1 1043839 gt meanprob62 1 388046 gt varprobe2 1 8123707 gt meanprobe3 1 3856897 gt varprobe3 1 796193 gt meanprobe4 1 4370326 gt varprobe4 1 3827925 35926 For in this example the least squares estimation for the pop ulation mean is 74047 Homework 6 Due Dec 14 930AM Solutions will be handed out exactly at 930AM So no late homework will be accepted 852 92 930 936 948 1216 1226 1 to Stat 312 Lecture 17 Two sample t test Moo K Chung mchung statwiscedu November 97 2004 Let X17 Xn and Y17 7Ym be two indepen dent samples from normal distributions with the same population variance7 ie gt var y 1 17025 gt var test x y 2 2 Xi N N01ng and N NOLng F test to compare two variances The pooled estimator of a is O 3975 alternative hypothesis true ratio of variance pvalue n 71S m 7 us 52 p nm72 is not equal to 1 95 percent confidence interv 002967475 342920369 sample estimates ratio of variances 04417034 The test statistic for testing HOCMXMY VS H13MX7 MY 7 7 The Welch Satterthwaite t test is an alternative to X 7 Y 7 1 X 7 lay l i l W N nlmig the two sample t test With equal variance7 and 1s 1 n m used when the assumption that the two populations Reject H0 if lTl gt tutgmlmlz have equal variances seems unreasonable The prob lem with this method is that it is asymptotically a t distribution 1 Example A study was conducted to compare the weights of cats and dogs Weights of cats 207 217 357 137 217 10 Weights of dogs 317 107 207 40 Assume normality and equal variance for both cats and dogs Is there any difference between the weights of cats and dogs gtttestxyalternativequottwosidedquot conflevel09 Welch Two Sample ttest datazx and y t 07073df 4778 pvalue 05124 alternative hypothesis true difference in means is not equal to 0 90 percent confidence interval gt xltc202135132110 gt ylt c31102040 gt Splt sqrt5varx3vary8 1 1052824 gt tmeanxmeanySpsqrt1513 20 361647 9 861647 gt t 1 0 6827026 Review problems Compute a CI for uX 7py in the gt 111005 8 previous Example Example 97 1 1 859548 Checking the equality of variance This topic will be discussed later in detail gt varx 1 75 2 1 N L 5 Stat 312 Lecture 05 Maximum Likelihood Estimation Moo K Chung mchungstatwisc edu September 14 2004 Invariance Principle If 6 is the MLE s of parameter 6 then the MLE of h6 is for some function h Proof partial Consider likelihood function L6 6satis es W 0 Let as me Then the likelihood function for b h6 is given by Lh 1 Differentiating the likelihood with respect to b we have Lh 1 7 dL6 d6 7 dL6 l d d6 das d6 h 6 Loglikelihood Maximizing L6 is equiva lent to maximizing ln L6 since In is an in creasing function Example This technique is best illus trated by nding the MLE of parameters in N M 02 l1 are the MLE of M and 02 respectively Note that 62 is not un unbiased estimator of 02 5 Additional problem previous midterm Let X17 X2 be a random sample from N07 16 Note that the sample size is 2 and the density function for X is 6 ex 76 2 m plt gt Find the likelihood function and use it to ob tain the maximum likelihood estimator of 6 Solution The likelihood function is L6 927r exp 7 9m 9592 Now get loglikelihood function lnL6 const ln6 7 6x Differentiate with respect to 6 we get dlnL6 l l Tgiiz z 0 Solving the equation we get 2z z Asymptotic unbiasness When the sample size is large the maximum likelihood esti mator of 6 is approximately unbiased The MLE of 6 is approximately the MVUE of 6 This is why it is the most widely used estima tion technique in statistics For the previous example Review problems Example 617 Example 618 Exercise 623 Exercise 629 Read Chap ter 7 Homework 11 Exercise 630 638 74 710 712 714 Due Sept 30 930am 7171 7171 13621143 2 oz ozasnaoo If explicit density function is not available you can not apply MLE In this case apply the method of moment matching Stat 312 Lecture 7 Con dence Intervals 111 M00 K Chung mchung statwiscedu February 117 2003 Concepts 1 If n is su iciently large7 approximate 1001 7 00 con dence interval for u is i i gumsW7 where 3 is the sample stan dard deviation 2 Let p denote the proportion of an individual with a speci ed property 1001 7 00 Cl for a population proportion p is i i 2012 l Qn39 3 One sided con dence interval An upper con dence bound for u is u lt i 2as and a lower con dence bound for u is u gtiizas In class problems EX Toss n 100 biased coins with PH p Suppose you observe 38 heads Construct 95 Cl of p rbinomn1p will generate a Bernoulli random sample of size n with PX 1 p gt Xlt rbinom100 1 04 33 49 65 81 97 O gt sdX 1 04878317 gt O38196049sqrt100 1 047604 gt 038 196049sqrt100 1 028396 Exercise 723 Use Concept 2 When 37 helmets were subjected to a certain im pact test7 24 showed damage Let p denote the proportion of helmets that would show damage under the test Find a 99 Cl for p Selfstudy problems Example 787 78 Use Concept 2 Stat 312 Lecture 19 Linear Regression M00 K Chung mchungstatwisc edu Nov 30 2004 Concepts 1 Let I be the speed of a car and y be the distance the car traveled in an hour hour Then we have model 9 50 5115 Suppose we have n paired measurements 1 yii 1 n Since all measurement are supposed to be noisy we introduce a noise term 6 in the above equation Our modi ed stochastic model is y o ir67 where e N N002 Since 6 is a random variable we use Y instead of y for convenience Y 50 511 6 Note that lEY 50 11 and WY 02 N Equivalently we can write the above linear model for each paired measurement 1 yj Yj 50 511139 6139 where yj is the observed value of random variable and Ej N 5 Note that 50 511 Let 5051 be estimators of 50 1 Then the predicted values or tted values are given by Y Aj 80 811quot The differences between the observations yj and the pre dicted values j are called the residuals errors ie Tjyjjyj oi irjn m Least squares estimation The least squares estimation is a method of estimating parameters 50 and 51 by min imizing the sum ofthe squarederrors SSE 55E Zltyr 2 291630 3111 j1 j1 j1 Then the regression line is given by y 80 811 By differentiating SSE with respect to 50 and 51 we get normal equations 50T51 O C O 0 gt O oo o O lol I I 60 70 80 90100 x T50 er l W Solving these equations we get A SW 51 Sm 30 y 7 7 where the sample covariance 51y nTy 7 i9 Example 10 students took two midterm exams Student 0102 03 04 05 06 07 08 0910 Midterm 1H 80 75 60 90 99 60 55 85 65 70 Midterm 2H 70 60 70 72 95 66 60 80 70 60 Let s nd the least squares regression line gt Xlt c80 75 60 90 99 60 55 85 65 70 gt ylt c70 60 70 72 95 66 60 80 70 60 gt plotxy gt rarara lt lmyx gt rarara Call lmformula y N X Coefficients Intercept X 294827 05523 gt ablinerarara Stat 312 Lecture 12 Errors in hypothesis testing M00 K Chung mchungstatwiscedu Oct 20 2004 Type I error you reject H0 when H0 is true PTypeIerror Preject HOIHO true Oz The resulting 04 is called the sigm eanee level of the test and the corresponding test is called a level 04 test We will use test procedures that give 04 less than a speci ed level 005 or 001 usually I believe that dogs are as smart as people Assume IQ ofa dog follows X N Nu102 IQ of 10 dogs are measured 30 25 70 110 40 80 50 60 100 60 We want to test if dogs are as smart as people by testing H0u100vsH1ult 100 1 Let X be a test statistic and R foo 90 to be a rejection region Let s compute the probabil ity of making Type I error based on this testing procedure Under the assumption H 0 is true X N N100102 Under this condition X N N1007 10 and a PX lt 90 gthelp pnorm Usage pnormq mean0 sd1 gt pnorm90100sqrt 10 l 00007827011 By using this test procedure it is highly unlikely to make Type I error A O V Let s see what happens when we change the test precedure When R foo 95 with the same test statistic X 04 PX lt 95 gt pnorm95100sqrt 10 l 005692315 2 e O C 5 0 11 0 g o 8 O I C C O C V I V mquot 039 E O39 E 2 0 CL 8 O I O O I I I I I O I I I I I 70 90 110 70 90 110 a a Figure 1 Errors in performing hypothesis testing based on test procedure if X lt 90 reject H0 ILL 100 Left gure type I error right gure type 11 error 3 WhenR 70099 a PX lt 99 gt pnorm99100sqrt 10 1 03759148 The test procedure based on rejecting H0 if X lt 99 is producing large Type I error Why Type II error you do not reject H0 when H0 is false 3 PType 11 error Pdo not reject HOIHO false 17 PrejectH0lH1 true With the xed test procedure of rejecting H 0 if X lt 90 let us compute B When ILL 90 is true gt 1 pnorm9090sqrt 10 l 05 gt 1 pnorm9095sqrt 10 1 09430769 gt l pnorm9099sqrt 10 1 09977867 Now let us see how 3 changes when we change the rejection region Assume u 90 is true gt alt 2020 gt alt a 90 Type II error gt right figure gt plotal pnorma90sqrt10type l Type I error gt left figure gt plotapnorma100sqrt10type l Reviewproblems Exercise 81 83 85 Stat 312 Fall 2003 09112003 Discussion 2 Poisson Process Let Pkt denote the probability that k pulses will be received by the counter during any particular time interval of length t 670 PW T where 04 is the expected number during a unit interval of time Then the number of pulses during a time interval of length it is a Poisson random variable with parameter A at Example The number of people arriving for treatment at an emergency room can be modeled by a Poisson process with a rate parameter of ve per hour a What is the probability that exactly four arrivals occur during a particular hour b What is the probability that at least four people arrive during a particular hour c How many people do you expect to arrive during a 45 min period Joint pdf and marginal pdf Let X and Y be continuous random variables with joint pdf7 fxy Then the marginal pdf of X and Y7 denoted by fXx and jfyy7 respectively7 are given by fX Lfydy for fooltxltoo fyy fxydx for 700 lt y lt 00 If X and Y are independent7 then ay ffy Example You have two lightbulbs for a particular lamp Let X the lifetime of the rst bulb and Y the lifetime of the second bulb both in 1000s of hours Suppose that X and Y are independent and that each has an exponential distribution with parameter A 1 a What is the joint pdf of X and Y b What is the probability that each bulb lasts at most 1000 hours ie7 X S 1 and Y S 1 c What is the probability that the total lifetime of the two bulbs is at most 2 Central Limit Theorem CLT Let X17 X2Xn be a random sample from a distribution with mean M and variance 02 Then if n is suf cient large7 X has approximately a normal distribution with M M and 0 039 Ting Li Lin 1 Equot E 4 Stat 312 Lecture 10 Con dence intervals for variance M00 K Chung mchungstatwiscedu August 13 2004 Suppose the fat content of a hotdog follows nor mal distribution 10 measurements are given gt XltC252213228170298210 255160209195 We are interested in constructing interval estimate of the unknown population variance To solve this problem we need to know the following fact Let X17 Xn be a random sample from N01700 7171 2 1 7 g 209 e X x3 j1 Quiz What is the expectation of xiil ylt 050 par mfrowc22 plot ydchisq y l type 1 plot ydchisq y 5 type 1 plot ydchisq y 10 type 1 plot ydchisq y20 type 1 VVVVVV Critical values for xgl distribution are de ned as numbers that gives PXiea2n lt Xi lt Xi2n1 a To nd Xagmg and 0253 that is need to con struct 95 CI for 02 we use R package gt qchisq00259 1 2700389 gt qchisq09759 1 1902277 10017 a CI for 02 n l 2 is XuzZmil 7 1 271 82 XliaZmil lt02lt l dchisqy 1 dchisqy 10 a 3 2 d 5 d 3 8 E 8 O 39U D o 20 40 o 20 40 y y 8 8 CT 0 2 o O J D d m g 6 0 20 40 o 20 40 y y Figure 1 The density functions of xi xg xi xgo re spectively Review problems Example 715 Excercise 745 Exercise 747 Additional problem for lecture 09 gt gt gt gt rbuJNH gt gt gt 1 1ibraryDevore6 dataex0747 attachex0747 exO747 strength 115 121 99 93 alt strengthgt10 a TRUE TRUE FALSE FALSE 1engtha 1 48 gt suma 1 13 Stat 312 Lecture 12 Testing on Population Proportion Moo K Chung mchung statwiscedu Feb 20 2003 Concepts 1 Testing mean u when the sample size is large HQZIUILLVS H12pltp0 E Mo Sx 39 level 04 test 2 S 720 Test statistic z Rejection region for 10 Let p be the proportion of a population with a speci ed property Large sample level 04 test for H0 11330 VS H13PltP0 P Po xpo17pon39 rejection region 2 lt 720 test statistic z 9quot The P value is the smallest level of signi cance at which H0 would be rejected P value S a a reject H0 at level 04 P value gt 04 a do not reject H0 at level 04 The smaller the P value the easier to reject H0 Inclass problems Example 811 47 out of 102 doctors did not know the generic name for the drug methadone Can you conclude that fewer than half of all doctors know the generic name for methadone Test it at 1 level gt plt 47102 gt p0 5 sqrt 0 50 5102 1 0792118 gt qnorm0 01 1 2326348 Assume the IQ of a dog follows normal The le of 10 dogs are measured 30 25 70 110 40 80 50 60 100 60 Find the P value for testing H0u100vsH1ult100 gt tmeanx100sdxsqrt10 gtt 1 4205156 gt qtc0001000200039 1 4296806 3834510 3572577 gt pt 42051569 1 0001144577 gt ttestxmu100alternativequotless conflevel095 Dne Sample ttest data x t 42036 df pvalue 0001147 alternative hypothesis true mean is less than 100 95 percent confidence interval Inf 788531 sample estimates mean of x 625 9 Three options for alternativezless greater two sided Find the P value for testing H0u100vsH1u7 100 Selfstudy problems Exercise 835 Example 817 Find the P value for testing H0u100vsH1ugt100 Assignment 4 Due March 13 1100am 824 826 828 836 a and 10 only 838 1 Equot 5 Stat 312 Lecture 08 Large sample con dence intervals M00 K Chung mchungstatwiscedu September 27 2004 The sample size is inversely related to the width of con dence interval Example 74 Central Limit Theorem Let X17 7Xn be a random sample with mean ILL and variance 02 For suf ciently large n X Z l N N0 1 a 71 Let X17 7Xn be a random sample with mean ILL For suf ciently large n 1 Z n N0 1 7 X 7 L S where S is the sample standard deviation If n is suf ciently large approximate 1001 7 a con dence interval for ILL is 7 s m i 20427 71 f where s is the sample standard deviation General large sample con dence interval Sup pose 0 is an unbiased estimator of some parame ter 0 Then 1001 7 a con dence interval is 717 2042 V V In many applications V is a function of 9 which makes computation of CI complicated In this sit uation we need to estimate V0 further Example Toss n 100 biased coins with PH p Suppose you observe 38 heads Con struct 95 CI ofp gt Xlt rbinom100 104 gt X l l 17 0 33 1 00000001011000 01101011010101 10000101000001 V39 49 01000011010110 65 00100100010010 8110100110001001 97 0 1 1 0 gt sqrt0381038100196 1 009513574 gt 0380095 1 0475 gt 0380095 1 0285 Onesided con dence interval An 1001 7 a upper con dence bound for 9 is 0 lt i 20 V W and a lower con dence bound for ILL is 0 gt i7zD V Review Problems Example 78 710 000 I OO I O Stat 312 Fall 2003 10302003 Discussion 8 1 z Test and Con dence Intervals for a Difference Between Two Population Means 11 Example 1 Exercise 2 page 370 Devore 6th Let 1 and 2 denote true average tread lives for two competing brands ofsize P20565R15 radial tires Test H0 M1 7 p2 0 versus H M1 7 pg 74 0 at level 05 using the following data m 45 i 42500 51 2200 n 45 7 40400 and 52 1900 12 Example 2 Exercise 8 page 370 Devore 6th Tensile strength tests were carried out on two different grades of wire rod 77Fluidized Bed Patenting of Wire Rods7 Wire J June 1977 56 61 resulting in the accompanying data Grade Sample Size Sample Mean kg7717712 Sample SD AlSl1064 m 129 i 1076 81 13 AlSl1078 n 129 y 1236 82 20 a Dose the data provide compelling evidence for concluding that true average strength for the 1078 grade exceeds that for the 1064 grade by more than 10 kgmmz Test the appropriate hypotheses using the P value approach b Estimate the difference between true average strengths for the two grades in a way that provides information about precision and reliability 2 The Two Sample t Test and Con dence Interval 21 Example 3 Exercise 18 page 378 Devore 6th Let 1 and 2 denote true average densities for two different types of brick Assuming normality of the two density distributions test H0 M1 7 p2 0 versus Ha M1 7 pg 74 0 using the following data m 6 i 2273 51 164 n 1 7 2195 and 52 240 22 Example 4 Exercise 32 page 32 Devore 6th The article cited in Exercise 34 in Section 73 gave the following summary data on pro portional stress limits for specimens constructed using two different types of wood Type of Wood Sample Size Sample Mean Sample SD Red oak 14 848 79 Douglas r 10 665 128 Assuming that both samples were selected from normal distributions carry out a test for hypotheses to decide whether the true average proportional stress limit for red oak joints exceeds that for Douglas r joints by more than 1 MPa Ting Li Lin Stat 312 Lecture 6 Con dence Intervals 11 M00 K Chung mchungstatwiscedu February 6 2003 Concepts 1 Let X N Nu02 with known 72 and unknown 2 1001 7 a con dence interval for u is Liiza2 a77 llUiZa2lagr 2 The sample size is inversely related to the width of con dence interval 3 Central Limit Theorem Let X1 Xn be a ran dom sample with mean u and variance 72 For large n 7 Z h N N071 0 n 4 Ifn is suf ciently large approximate 10017 a con dence interval for u is 2 i 2a25 where s is the sample standard deviation Inclass problems Continuing Exercise 625 construct 98 con dence in terval Example 74 Response time N Nult72 a 25 Find the sample size n that ensures 95 CI with a width of 10 Example 76 Alternating current AC voltage data gt dataxmp0706 gt attachxmp0706 gt strxmp0706 dataframe 48 obs of 1 variable Cl int 62 50 53 57 gt boxplotCl gt meanCl l 5470833 gt sdCl 1 5230672 gtmean Cl qnorm0 975 sd Cl sqrt length C1 1 532286 gtmean Cl qnorm0 975 sd Cl sqrt length C1 1 5618807 Ex Toss n 100 biased coins with PH 13 Sup pose you observe 38 heads Construct 95 CI of p gt Xlt rbinom100l04 gt X 1 17 33 49 65 81 97 0 gt sdX 1 04878317 gt sqrt038l 038lOO l l 004878317 gt 038l960049sqrt100 1 0389604 gt 038 1960049sqrt100 1 0370396 I OOI OH I OOI I OO I I I OOI O OOOOOI O OOOOOO I I OOI O I OI I OO OOI OI O OOOI I H OI I OOO I OOOI H OOI OOH OI I OI O I OOOOO I OOI I O I OI OOO Selfstudy problems Example 78 Exercise 713 719 725 In the above coin tossing example check if X is MLE Stat 312 Lecture 4 Maximum Likelihood Estimation M00 K Chung mchungstatwiscedu January 30 2003 Concepts 1 For a random sample X1 Xn the likeli hood function is given as the product of prob ability or density functions L fwi9fw29 fwn9gt The maximum likelihood estimatate 0f 0 maxi mizes L If we denote 6531 wn to be the maximum likelihood estimate The max imum likelihood estimator MLE 0f 0 is X1 7Xn Note at are numbers While X are random variables ie Equot W When the sample size is large the maximum like lihood estimator of 6 is approximately unbiased The MLE of 6 is approximately the MVUE of 6 This is Why it is the most widely used parameter estimation technique 5 If explicit density function is not available you can not apply MLE In this case apply the method of moment matching U Invariance Principle If l 2 are the MLE s of 61 62 the 1V1LE 0fhlt61762gt 1S hlt6162gt Inclass problems Example 615 Example 616 Example 621 Exercise 625 Assuming normal distribution nd 0 such that Pstrength S c 095 gt Xlt strength gt lengthx 1 10 gt sd X 1 1987852 Figure 1 Density of N07 1 gt sigmalt sqrtlengthx llengthxsdx gt sigma 1 1885842 gt qnorm095musigma 1 4154193 gt qnorm095 1 1644854 gt pnorm415musigma 1 09476644 gt ylt 303010 gt plotydnormy l Selfstudy problems Example 617 Example 618 Exercise 623 Exercise 629 Stat 312 Lecture 01 R Basics Moo K Chung mchungstatwiscedu September 2 2004 Histogram of a Frequency 10 15 20 25 30 35 Figure 1 Histogram for binge drinker data Data loading Let s illustrate basic R commands by following example 15 about binge drinking in college Data is the percentage of binge drinkers on 140 campuses across the United States gt library Devore6 gt data xmp0105 gtattachxmp0105 gt xmp0105 bingePct l 4 2 ll 3 13 138 67 139 67 140 68 2 Histogram Let s see some basic R commands W gt alt bingePct gt meana l 4233571 gt vara 1 2058361 gt sda l 1434699 gt hista To nd out more about hi st command use gt help hi st It will display a new window with detailed information about hi st Binge drinking percentage can be modeled sta tistically Let X is the binge drinking percent age at ith campus which is distributed normally with mean 40 and standard deviation of 14 ie X N N407 142 Since binge drinking percent ages in different campuses should be indepen dent Xi s should be assumed to be independent random variables X1 Xn form a random sample if X are independent and identically distributed random variables A statistic is a random variable Hence sample mean X ELI Xin is a statistic It will be distributed as X N N4O14O14214O Note Read section 54 55 and do EX548 552 563 Lecture 24 will be based on sections 61 62 l Equot E Stat 312 Lecture 06 Quantilequantile plots Moo K Chung mchungstatwiscedu September 23 2004 In order to compute 1001 7 a con dence interval it is required to nd 2012 that satis es PZ gt 2012 042 for given a We will study how to nd 2012 and more This lecture is based on Chapter 46 The pth quantile point q for random variable X is the point such that FqPX qp The textbook represent it in terms of percentile Note that p th quantile 100 x p th percentile So given p q F 19 For X N N07 1 it is easy to nd the pth qun tile using gt qnorml l Inf gt qnorm05 l 0 gt qnorm0 l Inf gt qnorm05 l 0 gt qnorm095 1 1644854 gt qnorm005 l l644854 In order to nd 20 we use command qnorm17a Given n observations m1 am we order them from the smallest to the largest and we have x 1 z n The ith smallest observation is de ned as the 7 05nth sample quantile point or 100i 7 05n sample percentile point gt library Devore6 gt data xmp0105 10 20 30 40 50 60 70 l Figure 1 Plot of ordered data bingePct showing sample sqth quantile V attachxmp0105 gt sq lt sort bincht 4 If bingePct really follows N427 142 then the sample quantiles should be resonably close to the corresponding quantiles of the normal dis tribution The corresponding quantile points for bingePct can be computed using gt qqnormll40 O5l4042l4 We can check how closely the sample quantiles corresponds to the normal distribution by plotting the quantile quantile plot QQplot of the sam ple quantiles vs the corresponding quantiles of a normal distribution Figure l gt plot q sq IfX N NW 02 then 2 X M N01 U 7