### Create a StudySoup account

#### Be part of our community, it's free to join!

Already have a StudySoup account? Login here

# STATISTCAL INFERNCE STAT 512

UW

GPA 3.66

### View Full Document

## 75

## 1

## Popular in Course

## Popular in Statistics

This 53 page Class Notes was uploaded by Providenci Mosciski Sr. on Wednesday September 9, 2015. The Class Notes belongs to STAT 512 at University of Washington taught by Michael Perlman in Fall. Since its upload, it has received 75 views. For similar materials see /class/192501/stat-512-university-of-washington in Statistics at University of Washington.

## Reviews for STATISTCAL INFERNCE

### What is Karma?

#### Karma is the currency of StudySoup.

#### You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!

Date Created: 09/09/15

STAT 542 Notes Winter 2007 MDP STAT 542 MULTIVARIATE STATISTICAL ANALYSIS 1 Random Vectors and Covariance Matrices 11 Review of vectors and matrices The results are stated for vectors and matrices with real entries but also hold for complex entries An m x n matrix A E aij is an array of mn numbers all am A aml amn This matrix represents the linear mapping E linear transformation A R gt Rm 11 a n gt A37 Where a E R is written as an n x 1 column vector and all aln 1 2211 aljmj AmE ERm aml amn ac 21lamjajj The mapping 11 clearly satis es the linearity property Aa by aAa bBy Matrix addition If A E aij and B E bij are m x n matrices7 then A 3 an bij Matrix multiplication If A is m x n and B is n x p7 then the matrix product AB is the m x p matrix AB whose ij th element is 12 ABM Educka 1671 Then AB is the matrix of the composition RP E R g Rm of the two linear mappings determined by A and B verify ABa ABac Va 6 RP STAT 542 Notes Winter 2007 MDP I ranspose matrix If A E aij is m x n7 its transpose is the n x m matrix A sometimes denoted by A Whose ij th element is aji That is7 the m row vectors n column vectors of A are the m column vectors n row vectors of A Note that verify 13 A By A H7 14 AB BA A I m x n7 B n x p A71 1471 A n x n7 nonsingular Rank of a matrix The row column rank of a matrix S m x n is the dimension of the linear space spanned by its rows columns The rank of A is the dimension r of the largest nonzero minor r x r subdeterminant of A Then verify row rankA column rankA rankA rankA rankAA rankAA row rankA S minm7n7 column rankA S minm7n7 rankA S minm7n7 row rankA m dim row spaceAt 7 column rankA n dim column spaceAi 7 Furthermore7 for A m x n and B n x 107 rankAB S minrankA7 rankB Inverse matrix If A n x n is a square matrix7 its inverse A 1 if it exists is the unique matrix that satis es AA l A lA L where I E In is the n x n identity matrix diagl7 7 1 If A 1 exists then A is called nonsingular or regular The following are equivalent 2 STAT 542 Notes Winter 2007 MDP a A is nonsingular b The n columns of A are linearly independent ie7 column rankA Equivalently7 Am y 0 for every nonzero a E R c The n rows of A are linearly independent ie7 row rankA Equivalently7 36 A y 0 for every nonzero a E R d The determinant A y 0 ie7 rankA De ne det geometrically Note that if A is nonsingular then A 1 is nonsingular and A l 1 A If A m x m and C n x n are nonsingular and B is m x 717 then verify rankAB rankB rankBC If A n x n and B n x n are nonsingular then so is AB 7 and verify 16 AB 1 34144 If A E diagcl17 7 cl with all di y 0 then A 1 diagcl1 17 7d1 Trace For a square matrix A E aij n x 717 the trace of A is 17 trA 211 am the sum of the diagonal entries of A Then 18 traA bB atrA btrB 19 trAB trBA Note A m x 717 B n x m 110 trA trA A n x n Proof of 19 trAB 21ABM 2211 aikbkigt 21 Walk 2me NBA 3 STAT 542 Notes Winter 2007 MDP Determinant For a square matrix A E aij n x 717 its determinant is IAIZ6ltWgtH1 VolumeA07 lln7 Where 7T ranges over all n permutations of 17 7 n and 7T l1 according to Whether 7T is an even or odd permutation Then 111 ABAB A7anxn 112 AE1 AE1 113 lAl E IAI A H771 aii if A is triangular or diagonal Orthogonal matrix An n x n matrix 1 is orthogonal if 115 FF I This is equivalent to the fact that the n row vectors of 1 form an orthonor mal basis for R Note that 115 implies that I F717 hence also I 1 I 7 which is equivalent to the fact that the n column vectors of 1 also form an orthonormal basis for R Note that 1 preserves angles and lengths7 ie7 preserves the usual inner product and norm in R for m y E R 7 Fm Py E FadF21 g3TTy 05y E 7 2 HM 2 E Fm F93 07 03 E 932 In fact7 any orthogonal transformation is a product of rotations and re ec tions Also7 from 113 and 1157 I 2 17 so F l1 STAT 542 Notes Winter 2007 MDP Complex numbers and matrices For any complex number 0 E a ib 6 C7 let 6 E a ib denote the complex conjugate of 0 Note that E c and cc a2 b2 E c27 a cd For any complex matrix C E cij7 let 639 Eij and de ne 0 C Note that rm cnrurcr The characteristic roots E of the n x n matrix A are the n roots l17 7 In of the polynomial equation LN m rnu These roots may be real or complex the complex roots occur in conjugate pairs Note that the eigenvalues of a triangular or diagonal matrix are just its diagonal elements By b for matrices with possibly complex entries7 for each eigenvalue 1 there exists some nonzero possibly complex vector u E Cquot st A lIu 07 equivalently7 118 Au la The vector u is called a characteristic vector E eigenvector for the eigenvalue l Since any nonzero multiple cu is also an eigenvector for l7 we will usually normalize u to be a unit vector7 ie7 u2 E uu 1 For example7 if A is a diagonal matrix7 say d1 0 0 0 d2 0 Adiagd177dn E 7 0 0 dn OT STAT 542 Notes Winter 2007 MDP then its eigenvalues are just d17 7cln7 with corresponding eigenvectors 1117 7un7 where 119 iEO0 1 00 u 7 7 7 7 7 7 is the i th unit coordinate vector Note7 however7 that in general7 eigenvalues need not be distinct and eigenvectors need not be unique For example7 if A is the identity matrix I 7 then its eigenvalues are 1771 and every unit vector u E R is an eigenvector for the eigenvalue 1 I u 1 u However7 eigenvectors u7 v associated with two distinct eigenvalues l7 m cannot be proportional if u cv then lu Au cAv cmv mu7 which contradicts the assumption that l y m Symmetric matrix An n x n matrix S E sij is symmetric if A AZ 167 Sij Sji Lemma 11 Let S be a real symmetric n x n matrix a Each eigenvalue l of S is real and has a real eigenvector y E R b If l y m are distinct eigenvalues of S with corresponding real eigenvec tors y and it then 7 J 1v ie yw 0 Thus if all the eigenvalues of S are distinct each eigenvalue l has exactly one real eigenvector 7 Proof a Let l be an eigenvalue of S with eigenvector u y 0 Then Su lu gt uSu luu l But 5 is real and symmetric7 so 5 S7 hence m uSu uSu uSu Thus uSu is real7 hence l is real Since 5 l is real7 the existence of a real eigenvector y for l now follows from b on p3 6 STAT 542 Notes Winter 2007 MDP b We have 57 l7 and Sew mw7 hence IWWWampF4WampWWwmmwmW soyw0sincel7 m I Proposition 12 Spectral decomposition of a real symmetric ma trix Let S be a real symmetric n x n matrix with eigenualues l17 7ln necessarily real Then there exists a real orthogonal matrix 1 such that am Srmn7 where D diagl17 7 ln Since SF I Dl7 the i th column vector 77 of 1 is a real eigenuector for l7 Proof For simplicity suppose that l17 7 ln are distinct Let 717 7777 be the corresponding unique real unit eigenvectors apply Lemma 11b Since 717 7 yn is an orthonormal basis for R717 the matrix 121 I Eyl77yn nxn satis es FT I7 ie7 1 is an orthogonal matrix Since each 77 is an eigenvector for li7 SF I D verify7 which is equivalent to 120 The case Where the eigenvalues are not distinct can be established by a perturbation argument Perturb S slightly so that its eigenvalues become distinct non trivial and apply the rst case Now use a limiting argument based on the compactness of the set of all n x n orthogonal matrices LI Lemma 13 IfS is a real symmetric matrix with eigenualues l17 7 ln77 122 mg 27117 7 123 S Hi1 17 Proof This is immediate from the spectral decomposition 120 of S I Positive de nite matrix An n x n matrix S is positive semi de nite psd also written as S 2 0 if it is symmetric and its quadratic form is nonnegative 124 26526 2 0 V36 6 R STAT 542 Notes Winter 2007 MDP S is positive de nite pd also written as S gt 0 if it is symmetric and its quadratic form is positive 125 ade gt 0 V nonzero x E R o The identity matrix is pd xIx gt 0 if x y 0 o A diagonal matrix diagd17 7 cl is psd pd iff each di 2 0 gt 0 0 HS n x n is psd7 then ASA is psd for any A m x n olsznxnispd7thenASAispdforanyAzmxnoffullrankmgn 0 AA is psd for any A m x n 0 AA is pd for any A m x n of full rank in S n Note This shows that the proper way to square a matrix A is to form AA or AA not A2 which need not even be symmetric o S pd gt 5 has full rank gt S71 exists gt S71 E 5715S 1y is pd Lemma 14 a A symmetric n x n matrix S with eigenvalues l17 7 ln is psd pd i each li Z 0 gt 0 In particular S Z 0 gt 0 ifS is psd pd so a pd matrix is nonsingular 1 Suppose S is pd with distinct eigenvalues l1 gt gt ln gt 0 and corre sponding unique real unit eigenvectors 717 gym Then the set 126 5 E x e R xS 1x 1 is the ellipsoid with principle axes mm my Proof a Apply the above results and the spectral decomposition 120 b From 1207 S I DZF with I y1yn7 so 5 1 I Dle and7 5 x e R Fmof ng 1 Fy Rquot lzDfly 1 2PM 2 2 11 In STAT 542 Notes Winter 2007 MDP But 50 is the ellipsoid with principal axes muh 7 mun recall 119 and I ui 7247 so 5 is the ellipsoid with principle axes mm 7 my I Square root of a pd matrix Let S be an n x n pd matrix Any n x n matrix A such that AA S is called a square root of S7 denloted by 5 From the spectral decomposition S I DZTV7 one version of SE is 127 5 Fdiagll77l 1 Ern r this is a symmetric square root of S Any square root 5 is nonsingular7 for 128 s S gt 0 Partitioned pd matrix Partition the pd matrix S n x n as n1 n2 5 S 129 S 1 11 12 712521 S22 7 where m n2 n Then both 511 and 522 are symmetric pd why7 512 5517 and verifyl 130 1m 512S 21 511 512 17111 0 5112 0 i 0 1772 521 S22 S 2 521 1772 0 S22 7 where 131 5112 E 511 5125231521 is necessarily pd why This in turn implies the two fundamental identities 132 511 512 1m 5125521 5112 0 If1 0 S21 S22 0 In2 0 522 5272 521 1712 7 133 SH 512 71 I711 0 517112 0 Im 512S 21 521 522 4231521 1 0 SJ 0 1m 7 The following three consequences of 132 and 133 are immediate 9 STAT 542 Notes Winter 2007 MDP 134 S is pd ltgt 5112 and 522 are pd ltgt 5221 and 511 are pd 135 S 5112Sgg Sgg1511 For a E 1 gt E R the quadratic form mS lm can be decomposed as 2 335713 1 SlgSilagSfltgaj1 5125272133 335272133 Exercise 15 Cholesky decompositions of a pd matrix Use 132 and induction on n to obtain an upper triangular square root U of S ie S UU Similarly S has a lower triangular square root L ie S LU Note Both U E Uij and L E lij are unique if the positivity conditions uii gt 0 Vi and lii gt 0 Vi are imposed on their diagonal elements To see this for U suppose that U U VV Where V is also an upper triangular matrix with each 11 gt 0 Then U 1VU 1V I so 1 E U lV is both upper triangular and orthogonal hence I diagl1 l1 D why Thus V U D and the positivity conditions imply that D I LI Projection matrix An n x n matrix P is a projection matrix if it is symmetric and idempotent P2 P Lemma 16 P is a projection matrix i it has the form f Im 0 137 PFlt0 0r for some orthogonal matrix 1 n x n and some in S n In this case rankP m trP Proof Since P is symmetric P I DZF by its spectral decomposition But the idempotence of P implies that each l 0 or 1 A permutation of the rows and columns which is also an orthogonal transformation may be necessary to obtain the form 137 I Interpretation of 137 Partition 1 as m n m 138 1 n r1 r2 10 STAT 542 Notes Winter 2007 MDP so 137 becomes 139 PF1F1 But I is orthogonal so FT In7 hence 140 rr E I In0mgt Thus from 139 and 1407 PF1 111110111 F17 PFQ 111110112 0 This shows that P represents the llnear transformation that projects R orthogonally onto the column space of F1 whlch has dlmenslon m trP Furthermore7 In P is also symmetric and idempotent verify with rankIn P n m In fact7 In P FF P 1121157 so In P represents the llnear transformatlon that projects R orthogonally onto the column space of F2 whlch has dlmenslon n m trIn P Note that the column spaces of F1 and F2 are perpendicular7 since IVng 0 Equivalently7 PIn P In PP 07 ie7 applying P and In P successively sends any a E R to 0 11 STAT 542 Notes Winter 2007 MDP 12 Matrix exercises 1 For 5 p x p and U p x q7 with S gt 0 positive de nite7 show that S UU S Iq US 1U7 where denotes the determinant and L1 is the q x q identity matrix 2 ForSszpandaszlwithSgt07showthat 71 71 i CLS a WW 3ForSszpandepxpwithSgt0andT207showthat i ATS 1 1 AiTS17 AiTS T 1 where A1 2 2 AP denote the ordered eigenvalues 4 Let A gt 0 and B gt 0 be p x p matrices with A 2 B Partition A as A11 A12 gt A A21 A22 and let A112 A11 A12A 21A21 Partition B in the same way and similarly de ne 312 Show 1 A11 2 B11 11 B 1 2 A l 14112 2 B112 5 For 5 p x p with S gt 07 partition 5 and 5 1 as i 511 512 1 i 511 512 S i 521 22 7 S i 521 522 7 respectively Show that 511 Z 5131 and equality holds iff 512 07 or equivalently7 iff 512 0 12 STAT 542 Notes Winter 2007 MDP 6 Now partition 5 and S71 as 511 512 513 S S S S21 522 523 E lt50 5912B 7 531 532 533 312 33 11 12 13 5 1 7 321 22 323 2 512 S123 i 531 532 533 5312 533 Then 5123 E 512 51235 315312 511 5135371531 512 5135371532 521 5235371531 522 5235371532 2 5113 5123 5213 5223 7 with similar relations holding for S 12 Note that 512S123717 512S123717 but in general 511 51112717 SH 511271 instead7 511 511423717 511 7g 5113923 1 Show 1 5123112 51123 5112 511371 5123522371 Sll71512 iv 511 2 5112 2 Slum When do the inequalities become equalities V 5123Sgg3 1 Sll 4 1512 4 for a 4 x 4 partitioning 13 STAT 542 Notes Winter 2007 MDP X1 13 Random vectors and covariance matrices Let X E be X71 a rvtr in Rquot The expected value of X is the vector EX1 1300 E E 7 EXn which is the center of gravity of the probability distribution of X in R Note that expectation is linear for rvtrs X 7 Y and constant matrices A7 B 7 141 EAX BY AEX BEY Zl Zln Similarly7 if Z E is a random matrix in Rmxn7 Zml 39 39 39 Zmn EZ is also de ned component wise EZ1 EZ1n EltZ s s EZm1 EZmn Then for constant matrices A k x m and B n x 107 142 EAZB AEZB The covariance matrix of X E the variance covariance matrix7 is CovX EX EXX EX VarX1 CovX17X2 CovX17Xn CovX27X1 VarX2 CovX27Xn CovXmX1 CovXmX2 VarXn 14 STAT 542 Notes Winter 2007 MDP The following formulas are essential for X n x 17 A m x n7 a n x 17 143 CovX EXX EXEX 144 CovAX b A CovX 14 145 VaraX b a CovX a Lemma 17 Let X E X17 7Xn be a random vector in R a CovX is psd b CovX is pd unless El a nonzero a E a17 7an E R st the linear combination aX E CLle CLan constant7 ie the support ofX is contained in some hyperplane of dimension 3 n 1 Proof a This follows immediately from 145 b If CovX is not pd7 then El a nonzero a E R st 0 a CovX a VaraX But this implies that a X const D For rvtrs X m x 1 and Yn x 17 de ne CoxX7 Y E X EXY EY CovX17Y1 CovX17Y2 COXX17 Yn CovX27Y1 CovX27Y2 COXX27 Yn CovXm7Y1 CovXm7Y2 CovXm7 Yn Clearly CovX7 Y CovY7X Then verify 146 CovX 1 Y CovX CovY l CovX7 Y 1 CovY7 X and verify X 11 Y gt CovX7Y 0 147 gt CovX 1 Y COVX CovY 15 STAT 542 Notes Winter 2007 MDP Variance of sample average sample mean of rvtrs Let X17 7X be iid rvtrs in RP7 each with mean vector p and covariance matrix 2 Set Xn X1 Xn Then EXn p and7 by 1477 148 CovXn Flz CovX1 X 2 Exercise 18 Verify the Weak Law of Large Numbers WLLN for rvtrs Xn converges to p in probability Xn g p7 that is7 for each 6 gt 07 Pm u S e gt 1 as n gt 00 Example 19a Equicorrelated random variables Let X17 7X be rvs with common mean u and common variance 02 Suppose they are eqm comelated7 ie7 CorXi7 Xj p Vi y j Let 149 X X1 Xn 33 2211Xi Xn2 the sample mean and sample variance7 respectively Then 150 EOE u so Xn is unbiased for 11 VarXn Va Xl X 712 7102 nn 1p02 why 151 07211 n 1p When X17 7Xn are uncorrelated p 07 in particular when they are independent7 then 1151 reduces to 0727 which gt 0 as n gt 00 When p y 07 however7 VarXn gt 02p y 0 so the WLLN fails for equicorrelated 239d ms Also7 151 imposes the constraint 152 3 p S 1 Next7 using 1517 1382 LE 2211 X3 X02 77 m M02 2 n 07211 n Up 2 153 1 002 HH 16 STAT 542 Notes Winter 2007 MDP Thus 8 is unbiased for 0721 if p 0 but not otherwise I Example 19b We now re derive 151 and 153 Via covariance matri ces7 using properties 144 and 145 Set X X17 7X77 so u 1 154 EX E pen7 where en n x 17 p 1 1 p p CovX 02 p p p 1 155 E 021 pIn penen Then Xn enX7 so by 1457 wmxn m mnpbn pn p712 since enen n 2 i 0711 01 UP which agrees with 151 To nd Esi7 write 21 21 XW a4m2 XWM Mm E X In anal X 156 E XQX7 where 6 E 1s a unit vector7 P E nez is the prOJection matrix of W rank 1 E tr ne7l that projects R orthogonally onto the 1 dimensional 17 STAT 542 Notes Winter 2007 MDP subspace spanned by en7 and Q E In nez is the projection matrix of rank n 1 E tr Q that projects R orthogonally onto the n 1 dimensional subspace e draw gure Now complete the following exercise Exercise 110 Prove Lemma 111 below7 and use it to show that 157 Earc220 n 1gtlt1 pm which is equivalent to 153 I Lemma 111 Let X n x 1 be a rvtr with EX 0 and CovX 2 Then for any n x n symmetric matrix A7 158 EXAX MAE 6046 This generalizes the relation EX2 VarX E X2 Example 196 Eqn 153 also can be obtained from the properties of the projection matrix Q First note that verify 159 Qen Q n 0 De ne Y1 160 YE f QX nx 17 Yn so 161 Em QEX we 0 BOW covm 0ng pm penele 162 021 pQ Thus7 since Q is idempotent Q2 Q7 EXQX EYY EtrYY trE YY am p tr c2 am pgtltn 1 18 STAT 542 Notes Winter 2007 MDP which again is equivalent to 153 I Exercise 112 Show that C0vX E 021 pIn penen in 155 has one eigenvalue 02 1 n 1 p with eigenvector en7 and n 1 eigenvalues 021 p I Exercise 113 Suppose that E C0vX n x n Show that the extreme eigenvalues of E satisfy A1Z 6331 Va1rCLX7 A742 HInHin1 VaraX D 19 STAT 542 Notes Winter 2007 MDP 2 The Multivariate Normal Distribution MVND 21 De nition and basic properties Consider a random vector X E X17 7Xp E RP7 where X17 7Xp are iid standard normal random variables7 ie7 Xi N N07 l7 so EX 0 and CovX Ip The pdf of X ie7 the joint pdf of X17 7Xp is me agreeawawwb 21 2wem7 mew For any nonsingular matrix A p x p and any h p x l E RP7 consider the random vector Y AX p Since the Jacobian of this linear actually7 af ne mapping is BB A gt 07 the pdf of Y is y 2WEA116EA71ElimyA lQi a IAAli e wim MAYlwim 22 2W 2767y7m 2 1y7m7 y 6 R197 where E0 AEX M M7 CovY ACovX A AA E E gt 0 Since the distribution of Y depends only on p and 27 we denote this distri bution by N190 7 E7 the multivariate normal distribution MVND on RP with mean vector p and covariance matrix 2 Exercise 21 a Show that the moment generating function of X is 1 23 mXw E Eew X 62w W b Let Y AX p where now A q x p and p E Rq Show that the mgf of Y is 24 myw E Eew Y ew w 2w where E E AA CovY Thus the distribution of Y E AX p depends only on p and 2 even when A is singular and or a non square matrix7 so we may again write Y N Nqp7 E 20 STAT 542 Notes Winter 2007 MDP Lemma 21 Af ne transformations preserve normality IfYNNqpE thehforC r x q and dzr x 17 25 ZECYdNNTCLd7 02C Proof Represent Y as AX u so Z CAX Cu d is also an af ne transformation of X 7 hence also has an MVND with EZ Cu d and C0VZ CACA 02C D Lemma 22 Independence ltgt zero covariance Suppose that Y N Npp7 E and partition Y M and E as P1 P2 101 Y1 p1 1 P1 Z311 2312 26 Y 2 p2 Y2 M 102 M2 102221 E22 7 Where P1 102 10 Then Y1 JL Y2 ltgt 212 0 Proof This follows from the pdf 22 or the mgf 24 I Proposition 23 Marginal amp conditional distributions are normal IfY N Npp7 E and 222 is pd then 28 Y1 I Y2 N Npl 1 2122272106 ML 23112 Y2 N Max127222 Proof Method 1 Assume also that E is nonsmgular By the quadratic identity 135 applied with p y7 and E partitioned as in 267 210 y ME lw M 31 M1 Z1222721y2 M22112 212 M22316 Since also Z 2112 Zgg7 the result follows from the pdf 22 21 STAT 542 Notes Winter 2007 MDP Method 2 By Lemma 21 and the quadratic identity 1327 Y1 2122272116 Ipl 2122 21 Y1 Y2 0 Ip2 Y2 N 1 212221M2 23112 0 P1P2 M2 7 0 222 Thus by Lemma 21 for C Ip1 0p1 X132 and 0192M1 m 7 respectively7 Y1 Z3122272136 N N171 M1 212221M27 Z31127 Y2 N N122 M27 2322 which yields 29 Also Y1 2122272116 1L Y2 by 211 and Lemma 227 so Y1 2122272116 Y2 N iv1 M1 212221M27 2112 which yields 28 D 22 The MVND and the chisquare distribution The chi square distribution xi with n degrees offreedom df can be de ned as the distribution of ZfZZEZZEZ27 Where Z E Z17 7Zn N Nn07In That is7 Z17 7Zn are iid stan dard N07 1 rvs Recall that 213 xi N Gamma oz g A 7 214 1305 7 215 VarXi 2n Now consider X N Nni7 E with E pd Then 216 Z E 2 12X M N Nn07In7 217 ZZ X MYE WX M N X3 22 STAT 542 Notes Winter 2007 MDP Suppose7 however7 that we omit 2 1 in 217 and seek the distribution of X My X n Then this will not have a chi square distribution in general Instead7 by the spectral decomposition E T DAIV7 216 yields X mX u 222 rzWrZ 218 E VDAV A1V12 Aan7 Where A17 7 An are the eigenvalues of E and V E FZ N Nn07 In Thus the distribution of X My X p is a positive linear combination of in dependent X rvs which is not proportional to a xi rV Check Via mgfsl Lemma 25 Quadratic forms and projection matrices Let er Nn 7 02 and let P be an n x n projection matrix with rankP trP E in Then the quadratic form determined by X f and P satis es 223 X 5gtPltX 5 02x3 Proof By Lemma 167 there exists an orthogonal matrix I n x n st i Im 0 P c p lt 0 0 p Then Y E FX f N Nn07 02117 so with Y Y17 7Yn7 X PX YltIS 8gtYY12Y73102X12W n 23 STAT 542 Notes Winter 2007 MDP 23 The noncentral chisquare distribution Extend the result 217 to 230 as follows First let Z E Z17 7 Zn N Nn 7In7 where f E 517 7fn E Rquot The distribution of Z12ZZ EZZE Z2 is called the noncentral chi square distribution with n degrees of freedom df and noncentrality parameter E27 denoted by Note that Z17 7 Zn are independent7 each with variance 17 but now 5 To show that the distribution of Z 2 depends on 5 only through its squared length 27 choose1 an orthogonal rotation matrix I n x n such that Ff EH707 707 ie7 I rotates E into EH707 70y7 and set Then the desired result follows since Z2 Y2 E 312 Y22 322 NlN1 71l2 lN1071l2 lN1071l2 E x 2 x x 224 E Xill5ll2Xi717 where the chi square variates in each line are mutually independent Let V E YE N xfw N N137 127 where 6 We nd the pdf of V as follows d fVU EPl S Y1 S l i d 1 W d1 x2739r 7V 1 7amp1 a M 7 e e e dt x2739r d1 7 e t 2dt 1 Let the rst row of F be 5 E and let the remaining n 1 rows be any orthonormal basis for Li 24 STAT 542 Notes Winter 2007 MDP k 1 6E d W 2 6724 6647612 127139 160 U 1 5 6k d W i2 e75 tgke Tdt why 127139 7 1 0 6 v 27Te 2 2k vk72e75 verify V k0 39 00 5k 12k71 73 5 v 2 e 2 225 6 5 Z 2 Ck 1 7 k0 2 2 F 122k Poisson weights pdf Of Xi2k where 2k k12552 k r 1 2k 27T Ck by the Legendre duplication formula for the Gamma function Thus we have represented the pdf of a xf6 rV as a mixture weighted average of central chi square pdfs with Poisson weights This can be written as follows 226 xf6 l K N xf2K where K N Poisson 62 Thus by 224 this implies that ZZ E Z 2 N xi6 satis es 227 xi6 l K N xi2K where K N Poisson62 That is7 the pdf of a noncehtml chi square m xi6 is a P0isson62 mixture of the pdfs of central chi square ms with n 2k df k 07 17 The representation 227 can be used to obtain the mean and variance of xi6 25 STAT 542 Notes Winter 2007 MDP ElXi5l EElXi2K Kl En 2K n 262 228 n 6 varixiwn Elvarltxi2K Kgt1variEltxi2K Kn E2n 2K Varn 2K 2n 462 462 229 2n 46 Exercise 26 Show that the noncentral chi square distribution xiw is stochastically increasing in both n and 6 I Next7 consider X N N710 7 E with a general pd 2 Then 230 XWX mmmm ximz lu since Z E E X N Nn2p71n and Wu mm Note that by Exercise 267 the distribution of XE lX in 230 is stochas tically increasing in n and 2371 p Finally7 let Y N Nnf7 02 and let P be a projection matrix with rankP m Then P FlFl Where Fll l Im cf 220 2227 so IIPYIIQ F1F1Y2 Firmrim YFlriY urnH2 But ray N Nmrgg aQPlrl NmF1 7 021m 26 STAT 542 Notes Winter 2007 MDP so by 230 with X Pal7 u F357 and E 021m7 HPYH2 FinHY N 2 STE15 2 PE 2 02 7 a2 Xm a2 Xm a2 Thus P 2 231 IIPYIIQ 02x3 24 Joint pdf of a random sample from the MVND Npu7 2 Let X17 7 Xn be an iid random sample from Npu7 2 Assume that E is positive de nite pd so that each Xi has pdf given by 22 Thus the joint pdf of X17 7Xn is f177n 67mi Zilmiu ZEN WEI 6 ZlltwrgtE 1ltwrm w 1 2u2 quotl l Iquot MyELIW Wrwn 2n 7 Z 5 232 2 Vilma67ltHgt21ltHgt7Mm7 7T 2 2 or alternatively7 71 271 23933 geniEilhi t x lt wows Where X s ioo XXXi X T ixixg il il il It follows from 232 and 233 that X75 and X7T are equivalent representations of the minimal su cient statistic for M723 Also from 2337 with no further restrictions on M72 this M VN statistical model constitutes a p MPH 439 39 quot7 full r quot7 family with natural parameter SAM7 2 1 27 STAT 542 Notes Winter 2007 MDP 3 The Wishart Distribution 31 De nition and basic properties Let X17 7 Xn be an iid random sample from Np07 E and set X X177Xn pgtltn7 SXXZXiX pgtltp i1 The distribution of S is called the pvariate central Wishart distribution with n degrees of freedom and scale matrix 27 denoted by lVpn7 E D Clearly S is a random symmetric positive semi de nite matrix with ES n2 When p 1 and E 027W1n702 U2 X31 Lemma 31 Preservation under linear transformation ForA qgtltp7 31 ASA N lVqn7 AXEY In particular for a p x 17 32 aSa N CLEa Xi Lemma 32 NonsingularityE positivede niteness of S N lVpn7 E S is positive de nite with probability one ltgt E is pd and n 2 p Proof Recall that S N XX with X p x n If n lt p then rankS rankX S minp7 n n lt p7 s0 5 is singular with probability one7 hence not positive de nite If E is not pd then Ela p x 17 a y 07 st aEa 0 Thus by 327 aSa N CLEa Xi 07 s0 5 is singular wprl 28 STAT 542 Notes Winter 2007 MDP lt2 Method 1 Stein Eaton and Perlman 1975 Ann Statist Assume that E is pd and n 2 10 Since 1 n s XX 2Xng Z XiXL 21 ipl it suf ces to show that 21 Xng is pd w pr 1 Thus we can take n 107 so X p x p is a square matrix Then S X27 so it su ices to show that X itself is nonsingular Wprl But 12 X singular 6 Si E span Xj j y i1 p PrX singular S ZP Xi 6 Si 11 19 ZEPrXi e Si IXm 21 0 11 since dimSi lt p and the distribution of Xi N Np07 E is absolutely continuous Wrto Lebesgue measure on RP Thus PrX nonsingular 1 lt2 Method 2 Okamoto 1975 Ann Statist Apply Lemma 33 Okamoto Let Z E Z17 7Zk E Rk be a random vector with a pdf that is absolutely continuous wrto Lebesgue measure on RI Let gz E 9217 7 zk be a nontrivial polynomial ie g E 0 Then 33 PrgZ 0 0 Proof sketch Use induction on k The result is true for k 1 since 9 can have only nitely many roots Now assume the result is true for k 1 and extend to k by Fubini7s Theorem equivalently7 by conditioning 011Z177Zk71 D Proposition 34 Let X p x n be a random matrix with a pdf that is absolutely continuous wrto Lebesgue measure on Rpm Ifn 2 p then 34 PrrankX p 17 29 STAT 542 Notes Winter 2007 MDP which implies that 35 PrS E XX is positive de nite 1 Proof Without loss of generality wlog assume that p S n and partition X as X17X2 with X1 p x p Since rankX1 lt p iff X1 07 and since the determinant X1 E gX1 is a nontrivial polynomial7 PrrankX1 p 1 by Lemma 33 But rankX1 p gt rankX p7 so 34 holds I Okamotols Lemma also yields the following important result Proposition 35 Let 15 2 2 135 denote the eigenvalues neces sarily real ofS E XXW Under the assumptions of Proposition 347 36 PrllS gt gt lpS gt 0 1 Proof sketch The eigenvalues of S E X X are the roots of the nontrivial polynomial hl E X X l Ip These roots are distinct iff the discriminant of h vanishes Since the discriminant is itself a nontrivial polynomial of the coef cients of the polynomial h7 hence a nontrivial polynomial of the elements of X 7 36 follows from Okamotols Lemma I Lemma 36 Additivity 1 51 JL 52 with Si N lVpni7 2 then 51 SQ N Wpn1 n27 30 STAT 542 Notes Winter 2007 MDP 32 Covariance matrices of Kronecker product form If X17 7X are independent rvtrs each with covariance matrix 2 p x 107 then CovX E X In7 a Kronecker product We now determine how a covariance matrix of the general Kronecker product form CovX E X A transforms under a linear transformation AX B see Proposition 39 The Kronecker product of the p x q matrix A and the m x n matrix B is the pm gtlt qn matrix A X B 2 f i A X B is bilinear 061141 062142 B 061141 062142 A iBl 52B2 mm o B1 52A 32 o o o H IL o o m S A B A 293 1 AA7BB gtA BA B iv If I p x p and l n x n are orthogonal matrices7 then I X l pn gtlt pm is orthogonal apply ii and iii v If A p x p and B n x n are real symmetric matrices with eigenvalues a1 7 ozp and 517 75m respectively7 then A X B pn gtlt pm is also real and symmetric with eigenvalues ai j i l7 7107 j l7 771 Proof Write the spectral decompositions of A and B as A FDOLF7 B IID5II7 31 STAT 542 Notes Winter 2007 MDP respectively7 where Da diagoz17 7 1p and D5 diag l7 75 Then A 69 B PDQFO ID xl 38 F 69 II Ba 69 DB F 69 w by ii andiii Since I X l is orthogonal and Doz 8 D5 is diagonal with diagonal entries ai j l l7707 j 177n7 38 is a spectral decomposition of the real symmetric matrix A X B 7 so the result followsLI Apsd7 Bpsd gtA Bpsd7 v1 A pd7 B pd gt A B pd apply 38 Let X E X17 7Xn p x n be a random matrix By convention we shall de ne the covariance matrix CovX to be the covariance matrix of the pn x 1 column vector X formed by stacking the column vectors of X X1 CovX1 CovX17Xn CovX CovX E Cov Xn CovXmX1 CovXn Lemma 37 Let X Xij E 01427 A Ajj Then COVX Z A ltgt COVij7 ny UiiAjj for all l7 2 l7 7p and all j7j l7 771 straightforward verify I Lemma 38 CovX E A ltgt CovX A8 2 Proof Set U X7 so Uij in Then COVUij7 Uij COVji7 Ujy UjjAii7 D hence CovX CovU A X E by Lemma 37 32 STAT 542 Notes Winter 2007 MDP Proposition 39 If COVX E X A then 39 C0V A X B ASA BAB vvv 5 h qgtltp pgtltn ngtltm qu me Thus ifX N NW 7 E A then 310 AXB N Nqu ACB7 AZY BAB Proof a Because AX Ale7 7AX it follows that A A 17 CovAX E covA7 A 1 com A Iny A In Z A A In AZY X A by b Next7 COVX A X 2 Lemma 387 COVBX BAB 2 b7 hence COVXB E COV BX E X BAB Lemma 38 Looking ahead Our goal will be to determine the joint distribution of the matrices 51127 5127 522 that arise from a partitioned Wishart matrix S In 34 we will see that the conditional distribution of 512 522 follows a multivariate normal linear model MNLM of the form 314 in 33 whose covariance structure has Kronecker product form Therefore we will rst study this MNLM and determine the joint distribution of its MLEs B7 given by 315 and5 16 This will readily yield the joint distribution of 51127 5127 522 which in turn will have several interesting consequences including the evaluation of ES 1 and the distribution of Hotelling s T2 statistic X254 33 STAT 542 Notes Winter 2007 MDP 33 The multivariate linear model The standard univariate linear model consists of a series X E X 17 7 X of uncorrelated univariate observations with common variance 02 gt 0 such that EX lies in a speci ed linear subspace L C R with dimL q lt n lf Z q x n is any xed matrix whose rows span L then 311 L Z 1quRq7 so this linear model can be expressed as follows E X Z7 1 x 7 312 52 5 2 q CovX 0 Im a gt 0 In the standard multivariate linear model7 X E X17 7Xn p x n is a series of uncorrelated p variate observations with common covariance matrix 2 gt 0 such that each row of EX lies in the speci ed linear subspace L C R This linear model can be expressed as follows EX5Z7 rpgtltq7 313 CovXE In7 Zgt0 If in addition we assume that X17 7Xn are normally distributed7 then 313 can be expressed as the normal multivariate linear model MNLM 314 XNPM Z7E In7 5mm 2gt0 Often Z is called a design matrix for the linear model We now assume that Z is of rank q S n7 so Z Z is nonsingular and is identi able 5 EX ZZZ 1 The maximum likelihood estimator B7 We now show that the MLE 57 exists w pr 1 iff n q 2 p and is given by 315 B XZZZ 17 316 3 ix In ZZZ 1ZX E xox 34 STAT 542 Notes Winter 2007 MDP Because the observation vectors X177Xn are independent under the MNLM 3147 the joint pdf of X E X17 7X is given by C 1 quot 007 71 0017 I f yz m 13 2 Et z zz 2 a 5Z1 Tff 67td2 122gwii znxri znH Z 5 317 3920 ei trP lwwzxxim 2 Where 01 2707 and Z17 7 Zn are the columns of Z To nd the MLES 37 37 rst x 2 and maximize 317 Wrto 5 This can be accomplished by minimizing the matrix valued quadratic form 318 Am 2 X ZXX ZY Wrto the Loewner ordem39ngg7 which a fortiom39 minimizes tr2 1A ver ify Since each row of Z lies in L E row spaceZ C R 7 this suggests that the minimizing 5 be chosen such that each row of Z is the orthogonal projection of the corresponding row of X onto L But the matrix of this orthogonal projection is P E zzzrlz n x n so we should choose 3 such that BZ XZZZ 1Z7 or equivalently7 319 B XZzzrl To verify that B minimizes AW7 write X Z X BZ f Z7 Am X BZMX BZY B mzzB m X BZWB m B ZX 82 0 0 2TZS T Sbmd 35 STAT 542 Notes Winter 2007 MDP Since Z Z is pd7 AW is uniquely minimized wr to the Loewner ordering when 5 57 so 320 m m Am X BZMX BZY X In zzzrllen zzzrlzy x E X In PIn PX EXQQX setQIn P XQX Q7 like P7 is a projection matrix Since 3 does not depend on 27 this establishes 315 Furthermore7 it follows from 317 and 320 that for xed 2 gt 07 Cl 71tr l 1anc max 339 i n 6 2 To maximize 321 w r to Z we apply the following lemma Lemma 310 IfW is pd then 672tr271W L 67 322 A 39239 ml 335 2 where fl E W is the unique maximizing value of 2 Proof Since the mappings E I gt 2 1 I A A H WAW 9 are both bijections of 8 onto itself7 the maximum in 322 is given by 1 n W lw W 1 WV 323 max A ei trmw Agt0 P 1 11w 7 n max wae 2 quot7 7 w122wpgt0 4 1 z 36 STAT 542 Notes Winter 2007 MDP where w17 7wp are the eigenvalues of 9 Since nlogw w is strictly concave in w7 its maximum value To is uniquely attained at t n7 hence the maximizing values of Lab 7 tap are 11 0p n Thus the unique maximizing value of Q is Q nIn7 hence A nlV 1 and i I If W is psdAbut singular7 then the maximum in 323 is Aoo verify Thus the MLE E for the MNLM 314 exists and is given by E XQX iff X QX is pd We now derive the distribution of X QX and show that 324 XQX is pd 111197 1 ltgt n q 210 Thus the condition To q 2 p is necessary and suf cient for the existence and uniqueness of the MLE E as stated in 316 First we nd the joint distn of B7 From 314 and 3107 M2 Q mm mm Q m 2 c2 prltqngtlt ZZ 072 ltZOZ ZQ0L from which it follows that 325 XZ NW ZZC 2 220 326 XQ NW 07 E Q7 327 XZ J XQ Because Q E In ZZZle is a projection matrix with verify rankltc2gt trQ n q its spectral decomposition is recall 137 LN 0 328 QFlt 0 0gtr for some p x p orthogonal matrix T Set Y XQI 7 so from 3267 In 0 MMQM Oq 0 37 STAT 542 Notes Winter 2007 MDP This shows that verify 329 XQX E YY N Wpn q7 Z7 hence 324 follows from Lemma 32 Lastly7 by 3257 3297 and 3277 330 B E XZZZr1 N NW 57 2 2204 331 mi E XQX N me q 2 332 B JL 2 Remark 311 From 3317 the MLE fl is a biased estimator of E Em 1 g 2 D Instead7 the adjusted MLE XV XQX is unbiased Special case of the MNLM a random sample from Np M7 2 If X17 7X is an iid sample from Np M7 2 then the joint distribution of X E X17 7 X is a special case of the MNLM 313 333 erNanQien7 2817 isz 17 Zgt0 Here q 17 Z en7 and Q In enenen 1en7 so from 330 3327 1 7 334 AXn n1 XiXnN is lt gt u eltenegt pom 335 n XQX 2 Xi X Xi X N me 17 2 i1 336 Xn JL i 38 STAT 542 Notes Winter 2007 MDP 34 Distribution of a partitioned Wishart matrix Let 8 denote the cone of real positive de nite p x p matrices and let Man denote the algebra of all real m x n matrices Partition the pd matriXSszpESIj as P1 P2 P1 511 512 337 S P2 521 5227 where p1 p2 p The next result follows from directly from 134 Lemma 312 The following correspondence is bijective s H s x MM x s 9 S lt gt 51127 5127 522 Note that we cannot replace 5112 by 511 in 337 because of the constraints imposed on S itself by the pd condition That is7 the range of 5117 5127 522 is not the Cartesian product of the three ranges Proposition 313 Let S N lVpn7 E be partitioned as in with n 2 p2 and 222 gt 0 Then the joint distribution of 51127 5127 522 can be speci ed as follows 339 512 l 522 N N171 x122 2312227215227 23112 522 340 522 N lVp2 n7 2227 341 5112 N Wp1n p27 21127 342 5127 522 JL 5112 Proof Represent S as YY with Y E N Nan 07 2 In7 s0 511 512 i YlYf Y1Y2 9 1smgtltnw g By Proposition 347 the conditions n 2 p2 and 222 gt 0 imply that rankY2 p2 w pr 17 hence 522 E Y2Y2 is pd w pr 1 Thus 5112 is well de ned and is given by 5112 Y1 E 39 STAT 542 Notes Winter 2007 MDP From 28 the conditional distribution of Y1 Y2 is given by 345 Y1 1 Y2 N NW 2122231167 2112 In which is a MNLM 314 with the following correspondences Xlt gtY17 5H2122 217 PHPh Zlt gtY27 2921127 qlt P2 Thus from 3257 3317 3327 3437 and 3447 conditionally on 127 346 512 l Y2 N N171 x122 2312227215227 23112 522 5112 Y2 N W171 n p27 21127 512 ll 5112 IYQ Clearly 346 gt 3397 while 340 follows from Lemma 31 with A 0p2Xpl Ip2 Also7 347 gt 341 and 347 gt 5112 11 127 which combines with 348 to yield 5112 11 5127 Y23 which implies 342 CI Note that 339 can be restated in two equivalent forms 349 5125231 1 522 N NM 21222317 2112 5231 l A 350 5125222 1 522 N NW 212223152227 2112 I where 52 can be any Borel measurable square root of 522 It follows from 350 and 342 that 11 212 0 gt 5125222 ll 522 ll 5112 We remark that Proposition 313 can also be derived directly from the pdf of the Wishart distribution7 the existence of which requires the stronger conditions To 2 p and Z gt 0 We shall derive the Wishart pdf in 84 Proposition 313 yields many useful results 7 some examples follow 3 Because A ll B I C and B ll C gt B ll A7 C verify 40 STAT 542 Notes Winter 2007 MDP Example 314 Distribution of the generalized variance IfS N lVpn7 E with n 2 p and Z gt 0 then 12 352 ISI N IEI HXiierh 11 a product of independent chi square variates Proof Partition 5 as in 337 with 101 17 p2 p 1 Then 5 5112 522 N W1n P 172112l Wp71n7222 N 2112X317p1 lid1971017 Z322M with the two factors independent The result follows by induction on p CI Note that 352 implies that although S is an unbiased estimator 0f 27 is a biased estimator of Ez 353 EESW EH 1n i lt E Proposition 315 Let S N lVpn7 E with n 2 p and Z gt 0 IfA q x 10 has rank q S p then 354 AS lAY1 Wq Tb 19 Az lAVl When A a 1 x 10 this becomes 1 1 2 355 N new Note Compare 354 to 31 ASA N lVqn7 AEA7 which holds with no restrictions on n7 107 27 A7 or q Our proof of 354 requires the singular value decomposition of A 41 STAT 542 Notes Winter 2007 MDP Lemma 316 IfA q x 10 has rank q S p then there exist an orthogonal matrix I q x q and a row orthogonal matrix Ill q x 10 such that 356 A FDA1 where Du diaga17 7 aq and a Z 2 a3 gt 0 are the ordered eigen values of 14144 By extending Ill to a p x p orthogonal matrix I E 1 gt7 2 we have the alternative representations 357 A F Du qupq II7 358 C Iq qupq II7 where C E I Da q x q is nonsingular Proof Let AA I DiF be the spectral decomposition of the pd q x q matrix AA Thus DglrAATDgl I q7 so Ill DglFA q x p satis es I lil Iq7 ie7 the rows of I1 are orthonormal Thus 356 holds7 then 357 and 358 are immediate I Proof of Proposition 315 It follows from 358 that verify 1197114qu C 1 11QC 17 AE lAfl 042112047 where 3 ISQI and 3 IXhll are partitioned as in 337 with 101 q and p2 p q Since 5 N lVpri7 E7 it follows from Proposition 313 that 3112 N Wq 71 P 17 21127 0713112071 N Wq 71 P 17 0712112071 D which gives 354 4 a1 2 2 Ch gt 0 are called the singular values of 42 STAT 542 Notes Winter 2007 MDP Proposition 317 Distribution of Hotelling s T2 statistic Let X N Npp7 E and S N lVpn7 E be independent n 2 p Z gt 0 and de ne T2 XS lX Then xiwE lu 359 T2 N X317121 E p nip1H271H7 a nonnormalz39zed noncentml F distribution The two chi square variates are independent Proof Decompose T2 as XE 1X By 355 and the inde pendence of X and S7 1 XS leXNXE 1X 2 7 Xn7p1 XS lX 1 XZilX N Xiierll independent ofX Since XE lX N Xian24 by 2307 359 holds I For any xed no 6 RP7 replace X and p in Example 317 by X p0 and p p07 respectively7 to obtain the following generalization of 359 T2 3 X MOYS VX 0 N xi M 01240 0 360 Fp n7p1 M lid2 10 0 2 Xnip1 Note In Example 611 and Exercise 612 it will be shown that T2 is the UMP invariant test statistic and the LRT statistic for testing 1 p0 vs 1 y to with 2 unknown When n 07 361 T2 D which determines the signi cance level of the test 43 STAT 542 Notes Winter 2007 MDP Example 318 Expected value of S71 Suppose that S N VVPW7 E with n 2 p and Z gt 07 so S71 exists with pr 1 When does ES 1 exist7 and what is its value We answer this by combining Proposition 313 with an invariance argument First consider the case 2 I Partition 5 and 5 1 as i 811 512 1 i 811 512 S 7 521 522 7 S 7 S21 S22 7 respectively7 with 101 1 and p2 p 1 Then by 3417 1 1 11 s N 2 7 8112 Xn7p1 SO 362 E311 7171371 lt 00 HT n 2 p 2 Similarly for the other diagonal elements of 5 1 lt oo iff n 2 p 2 Because each oil diagonal element 8 of 5 1 satis es lsijl S 18ii8jj S 8 we see that ES 1 A exists iff n 2 p 2 Furthermore7 because 2 I7 5 N PST for every p x p orthogonal matrix D hence FAF FES 1F E rsrrl ES 1 A vr Exercise 319 Show that FAT A VF gt A 6 for some 5 gt 0 I Thus ES 1 6L and 6 1 by 362 Therefore when E I7 nipil 13541 n2p2 Now consider the general case 2 gt 0 Since 44 STAT 542 Notes Winter 2007 MDP we conclude that 363 2 1 n 2 p 2 Proposition 320 Bartlett s decomposition LetSrvlVpn7 I withan SetSTT whereTEtij 1 Sj Sigp is the unique lower triangular square root ofS with tii gt 0 i 17 710 see Exercise 15 Then the tij are mutually independent rus with tfiwxiiiH 117707 364 tijNN10717 Proof Use induction on p The result is obvious for p 1 Partition 5 as in 337 with 101 p 1 and p2 1 so by the induction hypothesis7 511 TlTl for a lower triangular matrix T1 that satis es 364 with p replaced by p 1 Then 71 SE 511 512 lt Tlil B gt T1 Tll512gt ETT7 S21 822 S21T1 82221 0 32221 1 where T p x p is lower triangular with tii gt 07 i 17 710 Since T1 5121 and E I7 it follows from 3517 3507 and 341 with the indices 1 and 277 interchanged that Sngfl ll T1 ll 8221 521Tfl N N1xp7107 1 1294 2 8221 N Xn7p1 7 D from which the induction step follows 45 STAT 542 Notes Winter 2007 MDP Example 321 Distribution of the sample multiple correlation coef cient R2 Let S N lme7 E with n 2 p and Z gt 0 Partition 5 and E as l p l l p l 1 811 512 1 011 2312 365 S Z P 1ltS21 522 gt7 P 1lt221 2322 gt7 and de ne R2 7 5125221521 2 7 231222721221 7 7 p i 7 811 all 11 2 11 2 U i R2 i 51252721521 7 512522 gt 512522 gt 366 1 R2 8112 81 7 i p2 7 231222721221 Ci 1 p2 i 0112 2 2 15 242 VEV52272 12 22 22 22 21 0112 From Proposition 313 and 350 we have 7 51252212 52 N Nlgtltpil 212222152227 0112 Ip11gt7 8112 N 0112 Xip17 522 N Wp11n7 Z3227 8112 1 5127 522 so verify X12211V diitn 2 n7pl V N C Xi U i 522 N FpilynipHOL Therefore the joint distribution of U7 V E U7 VSgg7 2 is given by U IV N FpilynipjtlUL V N C Xi 46 367 STAT 542 Notes Winter 2007 MDP Equivalently7 if we set Z VC so Z is ancillary but unobservable7 then U l Z N Fp71n7p1CZ7 368 Z N X317 from which the unconditional distribution of U can be obtained by averaging over Z see Exercise 322 and Example A18 in Appendix A I Exercise 322 From A7 in Appendix A7 the conditional distribution Fp717n7p1CZ of U Z can be represented as a Poisson mixture of central F distributions 369 Fp17np1CZ lK N 1434th K N Poisson CZ2 Use 3687 3697 and A8 to show that the unconditional distribution of U resp7 R2 can be represented as a negative binomial mixture of central F resp7 Beta rvs 370 U l K N Fp712Knip17 U 7 n7 33971 RQEU HlK N Blt K7 S 7 372 K N Negative binomial p27 that is7 i F k 2 k 2 i 7 Note In Example 626 and Exercise 627 it will be shown that R2 is the LRT statistic and the UMP invariant test statistic for testing p2 0 vs p2gt0 Whenp20ltgt 2120 ltgt 0 UiLZby 368 and 373 U N Fp717n7p17 374 R2 N mg 71 D either of which determines the signi cance level of the test 47 STAT 542 Notes Winter 2007 MDP 4 The Wishart Density Jacobians of Matrix Transformations We have deduced properties of a Wishart random matrix S N lme7 E by using its representation 5 X X in terms of a multivariate normal random matrix X N Nan07 E X I We have not required the density of the Wishart distribution on 8 the cone of p x 10 positive de nite symmetric matrices In this section we derive this density7 a multivariate extension of the central chi square density Throughout it is assumed that n 2 p Assume rst that E I From Bartlett7s decomposition S TT in Proposition 3207 the joint pdf of T E Tij is given by verifyl 1 fit p 14 lt fT lgjll p We 2 J 2n7371rltn7Ti1gt 151 6 2 I 2 t3 ew 1 31 1 2 1951 s 2 CA tzii exp lt ltrTTgt 11 2 Since the pdf of S is given by fS fTl 7 we rst must nd the Jacobian ll E 1 ll of the mapping S TT This derivation of the Wishart pdf will resume in 44 41 Jacobians of vector matrix transformations Consider a smooth bijective mapping E diffeomorphism A B 42 gt E177n v gtyEy177yn7 Where A and B are open subsets of R The Jacobian matrix of this map ping is given by gi gamp a 001 001 43 z 7 a 600quot 600quot STAT 542 Notes Winter 2007 MDP and the Jacobian of the mapping is given by llT det Jaco bians obey several elementary properties Chain rule Suppose that a n gt y and y n gt z are diffeomorphisms Then a n gt z is a diffeomorphism and QHZBZ lgl 8m 7 yyx 83 no Proof This follows from the chain rule for partial derivatives aziy1a177an77yna177337 Z 823 am lt8zgtlt8ygt k ij 83 831k 83 8 3 83 Therefore g Z now take determinants I Inverse rule Suppose that a n gt y is a diffeomorphism Then 83 8y 1 45 H 83 yyw 833 Proof Apply the chain rule with z ac I Combination rule Suppose that a n gt u and y n gt v are unrelated diffeomorphisms Then lt46gt l3 lll Proof The Jacobian matrix is given by 801711 f lt 0 3amp7 y 0 3 39 Extended combination rule Suppose that 731 n gt U711 is a diffeo morphism of the form u Mac7 v U7 y Then 46 continues to hold Proof The Jacobian matrix is given by mwmgtlt gt may 0 3 39 49 STAT 542 Notes Winter 2007 MDP 42 Jacobians of linear mappings Let A p x p and B n x n be nonsingular matrices7 L p x p and M p x p be nonsingular lower triangular matrices7 U p x p and V p x p be nonsingular upper triangular matrices7 C a nonzero scalar A7 B7 L7 M 7 U7 V7 0 are non random Then 44 i 46 imply the following facts a vectors 3 0367 73 l x n combination rule b matrices Y cX7 X7Y p x n cp comb rule 6 symmetric matrices Y cX7 X7 Y 10x107 symmetric cpltp21 comb rule d matrices Y AX7 X7Y p x n comb rule YXB7X7Ypgtltn Bp comb rule YAXB7X7Ypgtltn AquotBp chain rule e symmetric matrices Y AXA7 X7 Y p x 107 symmetric 8Y APH ioX Proof Use the fact that A can be written as the product of elementary matrices of the forms Diagl7 7 1707 17 7 l7 l 0 0 1 Verify the result whenA andA Em then apply the chain rule I 50 STAT 542 Notes Winter 2007 MDP f triangular matrices 0 Y LX7 X7 Y p x 10 lower triangular or 1 i a H W 21 Proof Let Since yij 224 likmkj Z j7 the Jacobian matrix is 3311 3121 6 122 611717 1 l l 0011 60011 60011 60011 11 21 3311 3121 6 122 611717 0 1 pp 8 60021 60021 60021 60021 22 Y 7 3311 3121 6 122 691717 i 0 0 22 8X f 80022 80022 80022 80022 i 6 1111 3121 3122 6 Zpp 0 0 39 0 pp Bxpp Bxpp Bxpp Bxpp This is a pp 12 gtlt pp 12 upper triangular matrix whose determinant 112112 0 0 Y UX7 X7Yp x 10 upper triangular 8 p7i1 8X 39 P H luii 11 0 Y XL7 X7 Y p x 10 lower triangular or 1 1 la ll Ilal 21 Proof Write Y L X and apply the preceding case with U L I 5 A more revealing proof follows by noting that Y LX can be written column by column as Y1 1117 7 71 Lpo where Xi and are the p il X 1 non zero parts of the columns ofX and Y and where Li is the lower p i l X p i l principal submatrix of L Since LiXi has Jacobian lljjl the result follows from the composition rule 51 STAT 542 Notes Winter 2007 MDP 0 Y XU7 X7 Y p x 10 upper triangular oy 1 14 WH39 ii39 il Proof Write Y UX and apply the rst case with L U I 0 Y LXM397 X7 Y p x 10 lower triangular aY p i p 41 la H 1139 CH 39ml i39p 21 21 Proof Apply the chain rule I 0 Y UXV7 X7Y p x 10 upper triangular oy 1 4 1 4 W H luiilpiwrl 39 H lviilz 11 11 Proof Write Y VXU and apply the last case with L V and M U D g triangularsymmetric matrices 0 Y X X7 X p x 10 lower or upper triangular7 Y p x p symmetric D Proof Since yii 2mm 1 S i S 107 while yij 1341 3 j lt i S p 0 Y LX XL7 X p x 10 lower triangular7 Y p x p symmetric oy 1 4 P H1 axl 2 Iz Proof Clearly X n gt Y is a linear mapping To show that it is 1 1 LXl X1L LXg X L gt LXl X2 X1 X2L gt X1 X2L71 X1 X2L71 52

### BOOM! Enjoy Your Free Notes!

We've added these Notes to your profile, click here to view them now.

### You're already Subscribed!

Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'

## Why people love StudySoup

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "I bought an awesome study guide, which helped me get an A in my Math 34B class this quarter!"

#### "I was shooting for a perfect 4.0 GPA this semester. Having StudySoup as a study aid was critical to helping me achieve my goal...and I nailed it!"

#### "Their 'Elite Notetakers' are making over $1,200/month in sales by creating high quality content that helps their classmates in a time of need."

### Refund Policy

#### STUDYSOUP CANCELLATION POLICY

All subscriptions to StudySoup are paid in full at the time of subscribing. To change your credit card information or to cancel your subscription, go to "Edit Settings". All credit card information will be available there. If you should decide to cancel your subscription, it will continue to be valid until the next payment period, as all payments for the current period were made in advance. For special circumstances, please email support@studysoup.com

#### STUDYSOUP REFUND POLICY

StudySoup has more than 1 million course-specific study resources to help students study smarter. If you’re having trouble finding what you’re looking for, our customer support team can help you find what you need! Feel free to contact them here: support@studysoup.com

Recurring Subscriptions: If you have canceled your recurring subscription on the day of renewal and have not downloaded any documents, you may request a refund by submitting an email to support@studysoup.com

Satisfaction Guarantee: If you’re not satisfied with your subscription, you can contact us for further help. Contact must be made within 3 business days of your subscription purchase and your refund request will be subject for review.

Please Note: Refunds can never be provided more than 30 days after the initial purchase date regardless of your activity on the site.