APPL STAT EXP DESGN
APPL STAT EXP DESGN STAT 421
Popular in Course
Popular in Statistics
This 99 page Class Notes was uploaded by Providenci Mosciski Sr. on Wednesday September 9, 2015. The Class Notes belongs to STAT 421 at University of Washington taught by Scholz in Fall. Since its upload, it has received 25 views. For similar materials see /class/192507/stat-421-university-of-washington in Statistics at University of Washington.
Reviews for APPL STAT EXP DESGN
Report this Material
What is Karma?
Karma is the currency of StudySoup.
Date Created: 09/09/15
University of Washington Applied StatistiCs and Experimental Design Normal and Related Distributions I x2 and F Tests Power Sample Size amp Confidence Intervals Fritz Sgholz F aII Qluari eriZOQB Hypothesis Testing We have addressed the question Does the type of flux affect SIR Formally we have tested the null hypothesis H0 The type of flux does not affect SIR against the alternative hypothesis H1 The type of flux does affect SIR While H0 seems fairly specific H1 is open ended H1 can be anything but H0 There may be many ways for SIR to be affected by flux differences eg change in mean median or scatter Different effects for different boards Such differences may show up in the data vector 2 through an appropriate test statistic SZ Here 2 X1 X9Y1 Yg Test Criteria or Test Statistics In the flux analysis we chose to use the absolute difference of sample means sZ Y X as our test criterion or test statistic for testing the null hypothesis A test statistic is a value calculated from data and other known entities eg assumed eg hypothesized parameter values We could have worked with the absolute difference in sample medians or with the ratio of sample standard deviations and compared that ratio with 1 etc Different test statistics are sensitive to different deviations from the null hypothesis A test statistic when viewed as a function of random input data is itself a random variable and has a distribution its sampling distribution Sampling Distributions For a test statistic sZ to be effective in deciding between H0 and H1 the sampling distributions of sZ under H0 and H1 should be separable to some degree a 8 o m a c E m 2 0 g o39 o O O I I I I I I I 90 95 100 105 110 115 120 Sampling Distribution under Hg m N o39 o N gt o o 5 m 2 g o t n o 2 F E o m LO 0 o39 o O o I I I I 90 95 100 105 110 115 120 Sampling Distribution under H1 Sampled and Sampling Distributions Sampled Distributions distributions generating samples X1 MXn Sampling Distribution of Y under H0 Y Sampling Distribution of Y under H1 When to Reject H0 The previous illustration shows a specific sampling distribution for sZ under H1 Typically H1 consists of many different possible distributional models leading to many possible sampling distributions under H1 Under H0 we often have just a single sampling distribution the null distribution If under H1 the test statistics sZ tends to have mostly higher values than under H0 we would want to reject H0 when sZ is large as on the previous slide How large is too large Need a critical value Cent and reject H0 when sZ 2 Cerit Choose Cent such that PsZ 2 CeritH0 or a pre chosen significance level Typically x 05 or 01 It is the probability of the type I error The previous illustration also shows that there may be values sZ in the overlap of both distributions Decisions are not clear out gt type or type II error 5 Decision Table Truth Decision H0 is true H0 is false accept H0 correct decision type II error reject H type I error correct decision 0 Testing hypotheses like estimation is a branch of a more general framework namely decision theory Decisions are optimized with respect to penalties for wrong decisions ie PType Error and PType ll Error or the mean squared error of an estimate 9 of 6 namely E 62 relative frequency relative frequency 010 020 030 000 010 020 000 The Null Distribution and Critical Values accept H0 reject H0 type I error critical value 1049 significance level on 005 I I 90 95 100 105 110 115 120 Sampling Distribution under H0 accept H0 reject H0 type II error 90 95 100 105 110 115 120 Sampling Distribution under H1 Critical Values and pValues The p valuesz for the observed test statistic sz is PsZ 2 szH0 Note that p valuesz g x is equivalent to rejecting H0 at level x observed value 1071 p value 00097 5 m E 3 Ilca value 1049 Q E 005 2 I I 110 115 120 020 relative frequency 010 000 Sampling Distribution under Hi pValues and Significance Levels We just saw that knowing the p value allows us to accept or reject H0 at level or However the p value is more informative than saying that we reject at level or It is the smallest level or at which we would still have rejected H0 It is also called the observed significance level Working with predefined or made it possible to choose the best level or test Best Having highest probability of rejecting H0 when H1 is true This makes for nice and useful mathematical theory but p values should be the preferred way of judging and reporting test results for a specific test statistic It may well be that a different test statistic s1z gives a smaller p value than what the optimal test statistic s0z might give for a given 2 This complicates finding optimal tests based on p value behavior The Power Function The probability of rejecting H0 is denoted by 3 It is a function of the distributional model F governing 2 Le B 3F It is called the power function of the test When the hypothesis H0 is composite and when sZ has more than one possible distribution under H0 one defines the highest probability of type I error as the significance level of the test Hence x maximum3F F E H0 or limits the type I error probability For various F E H1 the power function gives us the corresponding probabilities of type II error as 1 Note that some people denote the probability of type II error by B 3F Thus make sure what is meant by 3F when you read or write about it Samples and Populations So far we have covered inference based on a randomization test This relied heavily on our randomized assignment of flux X and flux Y to the 18 circuit boards Such inference can logically only say something about flux differences in the context of those 18 boards To generalize any conclusions to other boards would require some assumptions judgement and ultimately a step of faith Would the same conclusion have been reached for another set of 18 boards What if one of the boards was an outlier board or something else was peculiar To be representative of the population of all boards we should view these 18 boards and their processing as a random sample from a conceptual population of such processed boards Conceptual Populations Clearly the 18 boards happened to be available at the time of the experiment They could have been a random sample of all boards available at the time However they also may have been taken sequentially in the order of production They certainly could not be a sample from future boards yet to be produced They could not be a sample of boards already in use on aircrafts The randomized processing steps might give the appearance of random samples assuming that these steps are mostly responsible for response variations Thus we could regard the 99 SIR values as two random samples from two very large or infinite conceptual populations of SIR values One sample of 9 boards from all boardsprocesses treated with flux X and one sample of 9 boards from all boardsprocesses treated with flux Y A board can only be treated with flux X or Y not both at the same time gt further conceptualization Population Distributions and Densities Such infinite populations of Z values are conveniently described by densities fz with the properties fz 2 0 and ffoofzdz 1 The probability of observing a randomly chosen element Z with Z g x is FxPZ xx fzdzx ftdt oo 00 z amp t are just dummy variables Avoid using x as dummy integration variable Fx as a function of x is also called the cumulative distribution function CDF of the random variable Z Fx from 0 to 1 as x goes from cgto to 00 For discrete populations with a finite or countany infinite number of distinct possible values z we replace fz by the probability mass function pz PZ z 2 0 and write Fx PZ g x Z pz with Zpz 1 ZSX Means Expectations and Variances The mean or expectation of Z or its population is defined by HMzEZ zfzdz or HEZZZPZ a probability weighted average of z values center of probability mass balance By extension the mean or expectation of gZ is defined by Eltgltzgtgt OO gZfZdZ or E 82 ZgZPZ Using gz z u2 the variance of Z is defined by 62 varltzgt E ltz m2 z H2fzdz or 62 2ltz ugt2pltzgt o oz varZ is called the standard deviation of Z or its population It is a measure of distribution spread Multivariate Densities or Populations fz1 zn is a multivariate density if it has the following properties fZ1zn20foralIZ1zn and fzlzndzldzn 1 It describes the behavior of the infinite population of such n tuples Z1 zn A random element 21Zn drawn from such a population is a random vector We say that Zl Zn in such a random vector are statistically independent when the following property holds fZ1739 quot751 f1Z1 X H anzn Here fiz is the marginal density of Z It is obtainable from the multivariate density by integrating out all other variables eg f2zz fZ1Z2Z3Zndzle3dZn E8Z17 Zn and COVZ17Z2 In analogy to the univariate case we define Eltgltzlzngt gltz1Zngtfltz1Zngtdz1dZn In particular using gZ1z2 zn Z1 Z2 EZ1zz Z1 Z2 fZ17Z2 dZ1dZ2 For independent 21 and 22 we have 52122 Z1 Z2 fZ17Z2 dZ1dZ2 Z1 Z2 f1 Z1f2Z2 dZ1dZ2 Z1 f1Z1 dZ1 Z2 f2Z2 dZ21 3Z1EZz Define the covariance of 21 and 22 as COVZ1722 EiZ1 EZ1 Zz EZzi EZ122 EZ1EZz Note that independent 21 and 22 gt cox2122 0 but not lt Also note cox2121 EZ Egg2 varZl Random Sample When drawing repeatedly values 21 Zn from a common infinite population with density fz we get a multivariate random vector 21Zn If the drawings are physically unrelated or independent we may consider 21Zn as statistically independent ie the random vector has density hZ17aZnfZ1X XfZn note f1fnf 21 Zn is then also referred to as a random sample from f We also express this as 21Zn quot1939 f Here iid independent and identically distributed 20 Rules of Expectations amp Variances Review For any set of random variables X1 Xn and constants a0a1 an we have provided the expectations EX1 EXn exist and are finite This holds whether X1 Xn are independent or not For any set of independent random variables X1 Xn and constants a0 a1 an we have vara0a1 gtltX1angtltXn agtltvaIX1a21gtltvarXn provided the variances varX1 varXn exist and are finite vara0 0 This is also true under the weaker than independence condition covXXj EXlXj EXlEXj 0 for 1 7E j In that case X1 Xn are uncorrelated 21 Rules for Averages 1 n 1 n 1 n 1 EXE 2X E EXZ 2 EXl 2yl1 n i n i n i n i 11 11 11 11 whether X1 Xn are independent or not If 1unu then EXu If X1 Xn are independent or uncorrelated we also have 1quot 1 n 1 n 1 n 2 12 varXVar ZXZ Zvar ZXi 2 2 vaIXi 225icn quot11 11 11 11 2 1 n 2 2 2 2 2 2 where 6n Zol 6no when 6126n26 n Snn 0 as n gt 00 provided 621 stays bounded eg 6n o A Normal Random Sample X1 Xn is called a normal random sample when the common density of the X is a normal density of the following form fx 2766 262 2 exp lt x 11 gt we also write X N Nyoz This density or its associated population has mean u and standard deviation 6 When u 0 and o 1 it is called the standard normal density 1 x2 CDF c1gt x d x eX with x z z ltigt m p 2 40 If X N Nyoz then Z X y6 N0 1 the standard normal distribution em 3 x PX G ac mo ltIgtltltx ugtogt 23 The CLT amp the Normal Population Model The normal population model is motivated by the Central Limit Theorem CLT This comes about because many physical or natural measured phenomena can be viewed as the addition of several independent source inputs or contributors YX1Xk or Ya l a1X1 l l aka for independent random variables X1 Xk and constants a0a1 ak Or in a 1 term Taylor expansion k M Hlv39unuk YfltX1Xkgt fltu1ukgtzltXi uigtT i1 l a0a1X1aka provided the linearization is sufficiently good ie the deviations X u are small compared to the curvature of f small oz 24 Central Limit Theorem CLT l Suppose we randomly and independently draw random variables X1 Xn from n possibly different populations with respective means 1 yn and standard deviations 61 6 Suppose further that the following variance ratio property holds 52 max 2 12 gt0 as n gtcgto i177n 616n ie none of the variances dominates among all variances obviously the case whenolonandn gtoo Then Y Yn X1 Xn has an approximate normal distribution with mean and variance given by uYu1ll un and 66 l 621 25 Central Limit Theorem CLT XI The last 4 slides illustrate the result when the condition none of the variances dominates is violated First we scaled up the log normal distribution by a factor of 10 and the resulting distribution of X1 X4 looks skewed to the right and looks in shape very much like the dominating log normal variation source In the second such example we instead scaled up the uniform distribution by a factor of 20 The resulting distribution of X1 X4 looks almost like a uniform distribution with somewhat smoothed shoulders In both cases the distribution with the dominating variability imprints its character on the resulting distribution of X1 X4 35 Derived Distributions from Normal Model Other than working with randomization reference distributions we otherwise generally assume normal distributions as the sources of our data Thus it is worthwhile to characterize some sampling distributions that are derived from the normal distribution They will play a significant role later on The chi square distribution the Student t distribution and the F distribution These distributions come about as sampling distributions of certain test statistics based on normal random samples Much of what is covered in this course could also be dealt with in the context of other sampling population models We will not get into that 38 Properties of Normal Random Variables Assume that X1 Xn are independent normal random variables with respective means and variances given by mun and 6621 Then Y X1 Xn N N011 uno o21 Geometric proof in Appendix A Here N means exactly distributed as If X N Nycz then a bX Ma 71117262 X U G N01 with 716 and a u6 Caution Some people write X N ame when others and I write X N NW 62 For example in R we have dnorm xy 6 pnorm xy S qnorm py 6 and rnorm my 6 which respectively give the density CDF quantile of Ny62 and random samples from NW 62 39 The ChiSquare Distribution When 21Zfi39139g39 N01 we say that f Cf Z 212 has a chi square distribution with f degrees of freedom i1 Memorize this definition We also write Cf N The density of Cf is 1 h x 2f2rn2 with mean f and variance 2f worth memorizing xf21exp x2 for x gt 0 not to memorize Density CDF quantiles of and random samples from the chi square distribution can be obtained in R via dchisq x f pchisq x f qchisq p f rchisq N f If Cf1 N xi and sz N 9th are independent then Cf1 sz N xgt fz since ZZJ1ZZ2 ngp fz with Z Zj independent N N01 4o density x2 Densities a df1 339 39 df2 H df5 2 M df20 g note how mean and variability increase with df H dfl G J d I I I HI I I I o 5 10 15 2o 25 30 41 The Nonoentral x Distribution Suppose X1 N Nd11Xf N adf 1 are independent Then we say that 2 2 2 has a noncentral 8 distribution with f degrees of freedom and noncentrality parameter 7 142 memorize definition EC f7u and vaIC 2f47u d1 df 0 0 gt previously defined central x3 distribution The distribution of C depends on d1 df only through A L1 l See geometric explanation on next slide What does R give us Forthe noncentralx2 we have dohisqx df nop0 pchisqq df ncp0 qchisqp df ncp0 rchisqn df ncp0 42 Dependence on d d2 7 Only 2 2 2 2 2 2 1 X X 1 x 7d x 7d densit contours of f xix2 exp 7 density contours of f xix2 exp 7 1 1 2 2 y 11211 2 IE This works because of the rotational invariance of density amp the blue circle region 43 density 015 010 005 000 Noncentral x2 Densities noncentral X A densities x x k x x M U l O OO 44 The Student tDistribution When Z N N0 1 is independent of Cf N x we say that Z C f f has a Student t distribution with f degrees of freedom We also write I N If Its density is Ff12 W mum2 It has mean 0 for f gt 1 and variance ff 2 if f gt 2 t memorize this definition for 00 lt x lt 00 not to memorize xzf1 f12 gfx gt px standard normal density as f gt 00 This follows either directly from the density using 1 x2ff2 gt eXp x22 or the definition of I because Cff gt 1 which follows from ECff1amp varCff 2ff2 gt 0 Density CDF quantiles of and random samples from the Student t distribution can be obtained in R via dt x f pt x f qt p f and rt N f respectively 45 density 04 03 02 01 00 Densities of the Student tDistribution df1 df2 df10 df20 df30 dfoo 46 The Nonoentral Student tDistribution When X N MRS 1 is independent of Cf N x we say that X iCff has a noncentral Student t distribution with f degrees of freedom and noncentrality t memorize this definition parameter nCp 5 We also write I N tfys R gives us dt x fncp pt q fncp qt p f ncp and rt r1 fncp respectively As before one can argue that this distribution converges to N51 as f gt oo 47 density 03 02 01 00 Densities of the Noncentral Student tDistribution df6ncp0 df6ncp1 df6ncp2 df6ncp4 These densities march to the left for negative ncp 48 density Densities of the Noncentral Student tDistribution V xix l 0 If 393 ncp4df 2 o quot3 quot k ncp4df 5 quot1quot H 4 I 391 g ncp4 df 2o ncp4df 30 o I I 5 0 5 10 15 t 49 The FDistribution When Cf1 N xi and sz N 9th are independent 8 random variables with f1 and f2 degrees of freedom respectively we say that F Cf1f1 szfZ has an F distribution with f1 and f2 degrees of freedom We also write F N Fflyfz lts density is memorize this definition Fllf1 f22 f1f2f12 lm 1 flXFf22lf1f2x1f1f22 not to memorize r Density CDF quantiles ofand random samples from the Ff1 7f2 distribution can be obtained in R via df x fl f2 pf x fl f2 qf p fl f2 rf N fl f2 respectively z3gt af WW NwFM1 RJ h WW 50 density 15 10 05 00 F Densities df11df23 df12df25 df15df25 df110df220 df120df220 df150df2100 51 The Nonoentral FDistribution Let C1 N xfhk be a noncentral 8 random variable and let C2 N 9 be a central 8 random variable which is independent of C1 then we say that C1f1 F NF CZfz f17f277 has a noncentral F distribution with f1 and f2 degrees of freedom and with noncentrality parameter 7 memorize definition What does R give us Forthe noncentralee have dfx dfl df2 ncp pfq dfl df2 ncp qfp dfl df2 ncp rfn dfl df2 ncp 52 density Noncentral F Densities noncentral F571oyldensities N U I O OO 53 Decomposition of the Sum of Squares SS We illustrate here an early example of the SS decomposition ix fort X43 i Xl X22Xl XXX2 11 11 i1 fog x iX XX it 11 i1 i1 fog ma i 2 iXl X2 l nX2 W i n N ll i x N ll i x since 2Xi X ZXi nX ZXi nZXin0 ie the residuals sum to zero Such decompositions are the intrinsic and recurring theme in the Analysis of Variance ANOVA to be addressed at length later 54 Sampling Distribution of X and 21Xi X2 When X1 Xn are a random sample from NW 62 the joint distribution of X and 2211 X X2 can be described as follows as is shown in Appendix B o X and 2211X X2 are statistically independent X N Mu zn 0r OhmONE x X HG M071 0 21Xi X252 N X21 0r 21Xi X2 N oleil ambiguously Only n 1 of the terms X X can vary independently since 2Xi X 0 The above independence seems perplexing given that X also appears within the expression 21X X2 It is a peculiar property of normal distribution samples This independence will not occur for other sampled distributions 55 OneSample tTest Assume that X X1 Xn i39lig39 MMGZ We want to test the hypothesis H0 u uO against the alternatives H1 u 7E u o is left unspecified and is unknown H0 is a composite hypothesis X is a good indicator for u since its mean is u and its variance is 62 ozn Thus a reasonable test statistic may be X u N ay yo ozn N0 ozn The last holds when H0 is true Unfortunately we do not know 6 V50 u0o X u0o N N01 suggests replacing the unknown 6 by suitable estimate to get a single reference distribution under H0 From the previous slide gt s2 1X X2n 1 N 62Cn1n 1 s2 is independent of X Note Es2 62 Le s2 is an unbiased estimate of 62 60 OneSample tStatistic Replacing o by s in the standardization O u0o gt one sample t statistic X HO og Hch og Hch Z Ntn1 since under H0 we have that Z V50 u0o N N01 and Cn1 x1214 both independent of each other We thus satisfy the definition of the t distribution zX Hence we can use tX in conjunction with the single known reference distribution tn1 under the composite hypothesis H0 and reject Hg for large values of tX The 2 sided level or test has critical value lcrit In171OC2 X2Il tCIit We reject H0 when zX 2 re t The 2 sided p value for the observed t statistic tobsx tobs is PIn 1ZIobsX 2Ptn 1 S tobsX 2 pt abst0bsan 1 61 Calculation of the Power Function of the TwoSided tTest The power function of this twosided t test is given by BUM P Z tGrit Pt g Zcrit Pt Z tGrit Pt g tcrit1Pt lttcrit r X XSO X Hgu MOo X G H HO5Z 5NI SS Cn1n1 n 18 noncentral t distribution with noncentrality parameter 5 g u0o delta Thus the power function depends on u and 6 only through 5 and we write B5 Pan 18 g Zcrit 1 PZn 18 lt tGrit pt tcritn 1delta 1 pttcritn 1de1ta The power function also depends on n 63 35 Power Function of TwoSided tTest sample size n 10 X 005 06 001 5J uuoc Where is the Flaw in Previous Argument We tacitly assumed that the power curve plot would not change with n ie we consider the effect of n only via 5 u u0o on the plot abscissa Both re t qt1 X2I1 1 and Ptn175 g item depend on n as does 5 u u0o See the next 3 plots Thus it does not suffice to consider the n in 5 alone However typically the sample size requirements will ask for large values of n In that case re t qnorm1 X2 and tn175 N51 stabilize for fixed 5 Compare n 100 and n 1000 in the next few plots For large n most of the benefit from increasing n comes via increasing 5 u u0o We will provide a function that gets us out of this dilemma 66 35 Power Function of TwoSided tTest sample size n 3 X 005 06 001 2 0 5J uuoc 67 Power Function of TwoSided tTest ammegmn30 35 2005 aom 4 2 0 5J uuoc 35 Power Function of TwoSided tTest sample size n 100 X 005 06 001 5J uuoc 69 35 Power Function of TwoSided tTest sample size n 1000 X 005 06 001 5J uuoc 70 power 09 Power of Two Sided t Test for Various n sample size n 73 Want power 9 at y yO 56 power 088 089 090 091 092 093 087 Power of TwoSided tTest for Various n re nedview LL HOVGu 05 090505 sample size n 74 n 44 will do for power 9 at u u0 56 Power Function of OneSided tTest wmmegmn3 65 a2005 aom 80 65 Power Function of OneSided tTest sample size n 30 x2005 x2001 81 65 Power Function of OneSided tTest sample size n 100 x2001 x2005 82 Power Function of OneSided tTest sampbsten1000 65 a2005 aom 83 power 10 09 08 07 06 05 o39wer of OneSided t Test for Various n 20 40 60 80 100 sample size n 85 Want power 9 at y 56 power Power of OneSided tTest for Various n re nedview 092 090 088 Ll HMO 05 on 005 086 30 32 34 36 38 40 sample size n 86 n 36 will do for power 9 at u u 56 Hypothesis Tests amp Confidence Intervals For testing H0 u u we accept HQ with the twosided t test whenever X u S ltln 11 oc2 ltgt HOEXiln 11 oc2gtlt Sx x Thus the interval X itn171a2 gtlt s consists of all acceptable uQ ie all u for which one would accept H0 u u at level or Furthermore since under H0 our acceptance probability is 1 x we have s s Pr X n 11 0c2 X E lt 0 lt Xln 11 oc2 X I 1 Here the subscript u on P indicates the assumed true value of the mean u Since this holds for any value u we may as well drop the subscript 0 on yo and write S S Pr X n 11 0c2XW lt H lt Xtn 11 0c2XW 1 1 The choice of lt or g is immaterial The case has probability zero 87 The Nature of Confidence Intervals S S X ln m ocz X W Xln 11 oc2 gtlt is called a 100 x 1 00 confidence interval for the unknown mean g It is a random interval which has probability 1 x of covering y This is not a statement about u being random due to being unknown or uncertain u does not fall into that interval with probability 1 or y is fixed but unknown Without knowing y we will not know whether the interval covers u or not Statistics means never having to say you re certain Myles Hollander lt Love means never having to say you re sorry Love Story by Eric Segal 88 What about OneSided Hypotheses Testing H u g go against H1 u gt uQ we reject when 30 u0s 2 tn171a or accept whenever WX HoSltln 11 oc ltgt HOgtX Zn 11 OLXS a ie X tn1710c gtlt s is a 1001 00 lower confidence bound for uO We could also state it in open ended interval form X tn1710c gtlt s po Clearly we would reject H0 whenever this interval shows no overlap with the interval 00 go as given by H0 y g yo Testing H0 u 2 yo vs H1 u lt uO reject when V50 u0s g tn17a or when the corresponding 1001 00 upper confidence bound interval 00 X tn17x gtlt s does not overlap the hypothesis interval Lug 00 93 Estimating 62 Xi X2 and s Have two estimates of 62 s 1 m Ms I ll H K How to combine or pool them si s2 or any other As 1 7 s si N 62x3n1m 1 gt vars2 G4 X 2m 1m 12 264m 1 Similarly vars 264n 1 and independence of s and s give us 264 264 2 1 1 x2 var As 1 As x a quadratic in 7 with clear minimum calculus exercise at 7 m 1mn 2 m 39 2 n 2 This suggests s2m 1S n1S12 i1Xl X 211YJ Y as best pooled variance estimate for 62 95 TwoSample tStatistic Thus 17 Y X6 W zXY SW SG Z Cmn 2m n 2 N tmn 2 gives us the desired 2 sample t statistic with known null distribution under H0 Reject H0 uy uX 0 at significance level or when IX Y Z tmn 21 oc2 1cm The 2 sided p value of the observed txy is Plmn 2 2 ZXaY 2 1 ptabstXYamn 2 97 Reflection on Treatments of TwoSample Problem Randomization test No population assumptions Under the hypothesis of no flux difference the SIR results for the 18 boards would be the same under all flux assignments The flux assignments are then irrelevant Test is based on random assignment of fluxes giving us the randomization reference distribution for calculation of p values or critical values Using tXY a good approximation to the null distribution often is tmn2 Generalization to other boards only by judgment or assumptions Normal 2 sample test Assumes 2 independent samples from normal populations with common variance and possibly different means We generalize upfront The t test makes inferences concerning these two conceptual populations 103 Density Density 04 02 00 02 04 00 FDistribution amp Critical Values for CC 05 degrees of freedom f1 8 f2 11 0c 0025 degrees of freedom f1 11 f2 8 0c 0025 2 0 1 2 3 Reciprocal of F Ratio 105 2 2 t Test when 5X 75 oY When 6 7E 6 we could emulate 17 X o er gn N N0 1 by using tXY 7 Xi s m l sn as test statistic But what is its reference distribution under H0 yX uy It is unknown This is referred to as the BehrensFisher problem Approximate the distribution of s erslzgn by that of a x xivf where a and f are chosen to match mean and variance of approximand and approximation a2 x 2f 2a2 f2 f 39 si s12 6 6 si s 2m 16 2n 16 2612 m n m n m m l n n l f EltagtltX3vfa and varltaxXf 107 Which Test to Use t or t When m n one easily sees that tXY tXY but their null distributions are only the same when s s in which case f 2m 1 However s s is an unlikely occurrence To show only takes some algebra What happens when m n but 6X 7E 6y and we use tXY anyway What happens when m 7E n and 6X 7E oy and we use tXY anyway How is the probability of type I error affected Such questions can easily be examined using simulation in R When m n or 6X 6y it seems that using tXYis relatively safe otherwise use tXY see following slides 109 Simulating the NullDistribution of tXY RecaH 12 12 Y X with S2m sX i n SY sHn1 n nytn Z zX Y and under H0 uX uy with independent Y X N N075n5 m7 m 1s 99314 and n 1s N 02 11 1 This leads to the first 3 command lines in the R function t sigdiff ie DbarrnormNsim0sqrtsigXAZmsigYA2n sZrchisqNsimm lsigXA2rchisqNsimn lsigYAZmn 2 tstatDbarsqrt52lmln 110 000 005 010 015 020 025 030 Inflated PType Error 6X21 GYZZ nx10 ny 5 5 0 5 conventional 2sample tstatistic 112 Density 05 04 03 02 01 00 Deflated PType Error 6X22 6Y21 nx10 ny5 4 2 o 2 4 conventional 2sample tstatistic 113 03 02 01 00 PType Error Hardly Affected GXZZO 6Y21 nx10 ny10 713 5 0 5 conventional 2sample tstatistic 115 Density 04 03 02 01 00 PType Error Mildly Affected 6X212 6Y21 nx10 ny5 conventional 2sample tstatistic 116 Density 04 03 02 01 00 PType Error Mildly Affected 6X21 6y12 nx10 ny5 conventional 2sample tstatistic 117 Checking Normality of a Sample The p quantile of N01 62 is xp u ozp Z is the standard normal p quantile Sort the sample X1 Xn in increasing ordeer g g X01 assigning fractional ranks pi E 01 to these order statistics in one of several ways for i 1 n i 5 i i 375 W n W Wm 0r Int H25 Plot X0 against the standard normal piquantile zpl qnormpi for i 1 n We would expect X0 xpl u ozpi ie X0 should look linear against zpl with intercept y and slope 6 Judging approximate linearity takes practice The third choice for pi is used by R in qqnorm x for a given sample vector x qqline x fits a line to the middle half of the data 118 Normal QQPlot n 16 V 139 o N N o o o O O o c o N N I I o 139 V I I I I I I I I I I I I I I I I I 4 2 4 2 0 2 4 4 2 0 2 4 4 2 2 lt1 V o N N o O O o o N N I 39 o lt1 lt1 I I I I I I I I I I I I I I I I I 4 2 4 2 0 2 4 4 2 0 2 4 4 2 2 Normal QQPlot n 64 ltr v N N o O 0 IV I I O 0 v ltr I I I 4 2 0 2 4 4 2 0 2 4 4 2 V V39 0 N N o o N N I O ltr ltr I I I I I I I I I I I I I I 4 2 0 2 4 4 2 0 2 4 4 2 Normal QQPlot n 256 ltr ltr ltr 0 N N N O O O N N N 0 ltf ltf V39 I I I 4 2 4 2 0 2 4 4 2 0 2 4 4 2 2 V V V39 0 0 N N N o o o N N N d O ltf ltf ltf I I I I I I I I I I I I I I I 4 2 4 2 0 2 4 4 2 0 2 4 4 2 2 EDFBased Tests of Fit Judgment We can also carry out formal EDF based tests of fit for normality Assume X1 Xn N G Test H0 Gx 613x uo for some u and o with u and o unspecified and unknown a composite hypothesis The empirical distribution function EDF Fnx is defined as 1 n Fnx Z 40 xX with 3x 1oo xX 1 or 0 as X1 gx or xltXl n 7 7 l Fnx proportion of sample values X1 Xn that are g x E success Here 31 x Bnx is an iid sequence of Bernoulli random variables with success probability p px PX g x Gx Law of Large Numbers LLN gt Bx Fnx gt Gx as n gt 00 for all x 122 q X HG enx and Gx 10 08 06 04 02 00 Empirical CDF for n 30 123 Empirical CDF for n 100 wo v0 NO 00 onzlxvveucoo 95 3d 124 Empirical CDF for n 30 with Estimated CDF 10 08 X HG 06 Fnx and Gx 04 02 00 II 50 5 5 n 30 sampled normal distr bution estimated normal distribution I I I I I I 40 45 50 55 60 65 126 Empirical CDF for n 100 with Estimated CDF O 1150 55 n100 00 O 6 3 I gtlt VG Fnx and Gx o4 sampled normal distribution estimated normal distribution 02 00 I I I I 3 I I I I I I I I I 127 Some Discrepancy Metrics Note the generally closer fit Gnx Fnx as compared to Gx m Fnx Fnx represents the sample and Gnx is fitted to the sample D1607 in sup in Kolmogorov Smirnov criterion X DCVMFnGn n Fnx Gnx2 nx dx Cram r von Mises criterion DAD Fm gnx dx Anderson Darling criterion Here nx is the density of Gnx ie nx px Xss where pz is the standard normal density 128 Computational Formulas for the Discrepancy Metrics D1607 in max mlax in GnXl mlax GnXl i 00an an 2 GnltXgt lt2i 1gtlt2ngt2 wow 11 made n lt1ngtlt2i 1gt Ioglt nltXgtgt loglt1 GnltXgtgt Here X0 g XQ g g X01 are the order statistics of the sample X1 Xn ie its values in increasing order Note that the distribution of GnXl CIXl Xs does not depend on the unknown parameters u and 6 since X X X 1 o X yo z z z z z with Z17 Zn1kd9 031 s so sZ 130 The Package nortest Fortunately the package nortest provides functions that evaluate each of the three discrepancy metrics and their corresponding p values Install the package nortest directly from the web or from the zip file nortestl 0 zip available on my class web site in your working directory Do this installation just once for each R installation Invoke library nortest for each R session during which you want to use it The package nortest contains the routines lillie test DKS cvmtest DCVM and adtest DAD Seedocumentation lillietest cvmtest and adtest 132 General Comments on GoodnessofFit GOF Tests Denote by H0 our distributional hypothesis here normality The following comments apply equally well to other distributional hypotheses For small sample sizes GOF tests tend to be very forgiving We reject only for gross deviations from H0 Very large samples from real applications most often lead to rejection of H0 The reason is that such tests are all consistent ie they will reject Hg for any alternative to H0 provided the sample is large enough Such alternatives may look very similar to H0 but not exactly the same Should we be concerned about such rejections Are tiny deviations from normality relevant The curse of large n GOF tests are most useful for moderate and not too large sample sizes 135 OCZ1BZ2NNO1 for 062B21 The normal convolution result on the previous slide follows by induction from the following special case which allows a simple and elegant proof 21 and 22 iid N M0 1 and a2 B2 1 gt 0ch 322 N M0 1 The crucial property that makes this proof possible is The joint density of 2122 has circular symmetry around 00 2 2 1 z z fZ17Z2 XP lt 1 2 7 21 2 Le points with same distance from 00 have the same density 138 Two Basis Representations of Z ZV1gtltf1V2gtltf2 szez 39 39 39 39 39 39 39 39 39 39 39 391 Z1gtlte122gtlte2 7 V1 Xf1 all points Z on this side ofthe red line have V1gta Z1gtlte1 all points Zion this side ofthe red line have ocZ1BZZ V1lt a 140 P0 Z1BZ2S0PZ1 S61 P0LZ1BZZ Z a PZ1 s a 141 Two Basis Representations of z V2Xf2k zv1gtltf1v2gtltf2 z1gtlte122gtlte2 with density 2 5 hzJJGXP Z Z 2 2 exp V V 2 hv 7 V1 X f1 Z1 gtlte1 146 Distribution of Z and 2Zl Z2 Suppose we choose 1quot 1 1 and choose orthonormal vectors for the other fl lt Gram Schmidt orthogonalization based on the basis f1e2 en Then Zf1V1f1 annf1 V1 ZZZ 2 is independent of 147 Review of GramSchmidt orthogonalization f1e2 en are a basis ofR since f1 1 1 7 212ae for any 622 an We get orthogonal basis vectors fl successively as follows f1 f1 and f2e1 a21f1 gt f 1f2f391e1 a21 0 gt a21f391e1 f3 e2 6131f1 6132f2 gt fllfg fllez a31ZO and flzf3 f392e2 a320 from previously constructed orthogonality f39lfz fquot2f1 0 thus 6131 fgez 1 12 Next f4 e3 a41f1 a42f2 a43f3 and multiplying this equation respectively by f f 2f 3 and setting to zero we get 6141 183 6142 283 and 6143 383 and SO on fi4 ffl i 1 n are then our orthonormal basis vectors 148 Applied Statistics and Experimental Design General Linear Model Fritz Scholz iFaII Quarter2l008 General Linear Hypothesis We assume the data vector Y Y1 YN consists of independent Y N mm 62 random variables tori 1 N We also have a data model hypothesis namely that the mean vector y can be any point in a given s dimensional linear subspace HQ C RN where s lt N Many statistical problem can be formulated as follows we identify a linear subspace Hm of Hg of dimension s r with 0 lt r g s and we test the hypothesis H0 u E Hm against the alternative H1 Iu E HQ Hm To distinguish Hm from HQ we may want to call Hm the test hypothesis LSE s MLE s Maximum Likelihood Estimates Under the normal distribution model for the Y the LSE s are also the maximum likelihood estimates MLE s of 1 wrt to the respective model constraints y E HQ and y E Hm This follows immediately from the likelihood function for the observed Yyy177yNl 1 N i i2 L7Gfoy17ayN6 2 6X13 lt W which is maximized over HQ Hm by minimizing 105 ul2 wrt u E HQHD and by taking 62 lm ampl2N and 32 lm f12N respectively