LINEAR STATISTICL MODELS
LINEAR STATISTICL MODELS STAT 714
Popular in Course
Popular in Statistics
verified elite notetaker
This 6 page Class Notes was uploaded by Shane Marks on Monday October 26, 2015. The Class Notes belongs to STAT 714 at University of South Carolina - Columbia taught by I. Dryden in Fall. Since its upload, it has received 30 views. For similar materials see /class/229648/stat-714-university-of-south-carolina-columbia in Statistics at University of South Carolina - Columbia.
Reviews for LINEAR STATISTICL MODELS
Report this Material
What is Karma?
Karma is the currency of StudySoup.
You can buy or earn more Karma at anytime and redeem it for class notes, study guides, flashcards, and more!
Date Created: 10/26/15
STAT714 LINEAR STATISTICAL MODELS Fall Session 2009 Handout 2 More on Projection matrices Result M15 If M is a perpendicular projection matrix onto CX7 then CM Proof We need to show that CM Q CX and CX Q Suppose that V E Then7 V Mb7 for some b Now7 write b b1 b27 where b1 6 CX and ngCX Thus v Mb Mb1 b2 Mbl sz b1 6 CX Thus CM g CX Now suppose that V E Since M is a perpendicular projection matrix onto CX7 we know that V MV MV1 V27 where V1 6 CX and VZICX But7 MV1 V2 MV17 showing that V E Thus7 CX Q CM and the result follows E Result M16 Perpendicular projection matrices are unique Proof Suppose that M1 and M2 are both perpendicular projection matrices onto any arbitrary subspace S Q R Let V E R and write V V1 V27 where V1 6 S and VZIS Since V is arbitrary and M1V V1 M2V7 we have M1 M2 D Result M17 If M is the perpendicular projection matrix onto CX7 then I 7 M is the perpendicular projection matrix onto Sketch of Proof I 7 M is symmetric and idempotent so that I 7 M is the perpendicular projection matrix onto CI 7 Show that CI 7 M NXT and use Result M16 D STAT714 LINEAR STATISTICAL MODELS Fall Session 2009 Handout 4 Quadratic forms de niteness and factorizations A11 Quadratic forms de niteness and factorizations TERMINOLOGY Suppose that x is an n gtlt 1 vector A quadratic form is a function f R a R of the form n n fx Z Z Jigmy XTAX i1 j1 The matrix A is called the matrix of the quadratic form Result M26 lf XTAX is any quadratic form7 there exists a symmetric matrix B such that XTAX XTBX Proof Note that XTATX XTAXT XTAX7 since a quadratic form is a scalar Thus7 1 1 XTAX XTAX XTATX 1 1 XT 7A iAT x XTBX7 2 2 where B A AT It is easy to show that B is symmetric D UPSIIOT In working with quadratic forms7 we can7 without loss of generality7 assume that the matrix of the quadratic form is symmetric TERMINOLOGY The quadratic form XTAX is said to be 0 nonnegative de nite nnd or positive semide nite psd if XTAX 2 07 for all x E R 0 positive de nite pd if XTAX gt 07 for all X 31 0 TERMINOLOGY A symmetric n gtlt 71 matrix A is said to be nnd or pd if the quadratic form XTAX is nnd or pd7 respectively Result M27 Let A be a symmetric matrix Then 1 A pd gt A gt 0 2 A nnd gt A 2 0 Result M28 Let A be a symmetric matrix Then 1 A pd ltgt all eigenvalues of A are positive 2 A nnd ltgt all eigenvalues of A are nonnegative Result M29 A pd matrix is nonsingular A nnd matrix with zero determinant is singular The converses are not true Result M30 Let A be an m gtlt 71 matrix of rank 7 Then ATA is nnd with rank 7 Furthermore ATA is pd if r n and is nnd with zero determinant if r lt 71 Proof Let x be an n gtlt 1 vector Then XTATAX AXTAX 2 0 showing that ATA is nnd Also rATA rA r by Result M3 If r n then the columns of A are linearly independent and the only solution to Ax 0 is x 0 This shows that ATA is pd If r lt n then the columns of A are linearly dependent ie there exists an x 31 0 such that Ax 0 Thus ATA is nnd but not pd D OHOLESKY FAOTORIZATION A square matrix A is pd iff there exists a nonsingular lower triangular matrix L such that A LLT Monahan proves this result see pp 258 provides an algorithm on how to nd L and includes an example SYMMETRIO SQUARE ROOT DECOMPOSITION Suppose that A is symmetric and pd Writing A in its Spectral Decomposition we have A QDQT Because A is pd A1 A2 A the eigenvalues of A are positive Result M28 If we de ne AlZ QDlZQT where DlZ diagT1 0 2 VA then A12 is symmetric and A12A12 QDiZQTQDiZQT QDileiZQT QDQT A The matrix A12 is called the symmetric square root of A STAT714 LINEAR STATISTICAL MODELS Fall Session 2009 Handout 3 Eigenvalues and eigenvectors A10 Eigenvalues and eigenvectors EIGENVALUES Suppose that A is a square matrix and consider the equations Au Au Note that AuAultgtAu7AuA7AIu0 If u 31 07 then A 7 AI must be singular Thus7 the values of A which satisfy Au Au are those values where A 7 AI 0 This is called the characteristic equation of A If A is n gtlt 717 then the characteristic equation is a polynomial in A of degree n The roots of this polynomial7 say7 A17 A27 WA are the eigenvalues of A some ofthese may be zero or even imaginary If A is a symmetric matrix7 then A17A277An must be real Searle7 pp 290 EIGENVECTORS lf A17 A27 7 An are eigenvalues for A7 then vectors ul satisfying Alli Mun for 239 1727 quot7717 are called eigenvectors Note that Aul Aiui gt Aul 7 Aiui A 7 AiIui 0 From our discussion on systems of equations and consistency7 we know a general solution for ul is given by ul I 7 A 7 AiI A 7 AlIz7 for z E R Result M20 lf AZ and Aj are eigenvalues of a symmetric matrix A7 and if AZ 31 A73 then the corresponding eigenvectors7 ul and uj7 are orthogonal Proof We know that Aul Aiui and Au Ajuj The key is to recognize that T 7 T 7 T Aiui u 7 ul Au 7 Ajul u7 which can only happen if AZ Aj or if ulTuj 0 But AZ 31 Aj by assumption D PUNOHLINE For a symmetric matrix A7 eigenvectors associated with distinct eigenvalues are orthogonal we7ve just proven this and7 hence7 are linearly independent MULTIPLIOITY If the symmetric matrix A has an eigenvalue Ak7 of multiplicity mk then we can nd mk orthogonal eigenvectors of A which correspond to Ak Searle7 pp 291 This leads to the following result cf7 Christensen7 pp 402 Result M21 If A is a symmetric matrix7 then there exists a basis for CA consisting of eigenvectors of nonzero eigenvalues If A is a nonzero eigenvalue of multiplicity m7 then the basis will contain m eigenvectors for A Furthermore7 NA consists of the eigenvectors associated with A 0 along with 0 SPECTRAL DECOMPOSITION Suppose that Amm is symmetric with eigenvalues A17 A27 7 An The spectral decomposition of A is given by A QDQT where 0 Q is orthogonal ie7 QQT QTQ I7 0 D diagA1A27 An7 a diagonal matrix consisting ofthe eigenvalues of A note that rD 7 A7 because Q is orthogonal7 and o the columns of Q are orthonormal eigenvectors of A Result M22 If A is an n gtlt n symmetric matrix with eigenvalues A17 A27 7 An7 then 1 lAl Hi i 2 trA 21 Ai NOTE These facts are also true for a general 71 gtlt 71 matrix A Proof in the symmetric case Write A in its Spectral Decomposition A QDQT By Result M19 lAl lQDQTl lDQTQl lDl 1 1 A By Result M18 trA trQDQT trDQTQ trD 221 A D Result M23 Suppose that A is symmetric The rank of A equals the number of nonzero eigenvalues of A Proof Write A in its spectral decomposition A QDQT Because rD rA and because the only nonzero elements in D are the nonzero eigenvalues the rank of D must be the number of nonzero eigenvalues of A D Result M24 The eignenvalues of an idemptotent matrix A are equal to 0 or 1 Proof If A is an eigenvalue of A then Au Au Note that A2u AAu AAu AAu A2u This shows that A2 is an eigenvalue of A2 A Thus we have Au Au and Au A2u which implies that A 0 or A 1 D Result M25 If the n gtlt 71 matrix A is idempotent then rA trA Proof From the last result we know that the eignenvalues of A are equal to 0 or 1 Let V1V2 V be a basis for CA Denote by S the subspace of all eigenvectors associated with A 1 Suppose V E S Then because AV V E CA V can be written as a linear combination of V1 V2 VT This means that any basis for CA is also a basis for 8 Furthermore NA consists of eigenvectors associated with A 0 because AV 0V 0 Thus 71 dimR dimCA dimNA rdimNA showing that dimNA n 7 r Since A has n eigenvalues all are accounted for A 1 with multiplicity r and for A 0 with multiplicity n 7 r Now trA A r the multiplicity of A 1 But rA dimCA r as well D
Are you sure you want to buy this material for
You're already Subscribed!
Looks like you've already subscribed to StudySoup, you won't need to purchase another subscription to get this material. To access this material simply click 'View Full Document'