(The QR Method) This problem outlines the basic ideas of

Chapter , Problem 8

(choose chapter or problem)

(The QR Method) This problem outlines the basic ideas of an alternative method, the QR method, of finding the least squares estimate \(\hat{\boldsymbol{\beta}}\). An advantage of the method is that it does not include forming the matrix \(\mathbf{X}^T \mathbf{X}\), a process that tends to increase rounding error. The essential ingredient of the method is that if \(\mathbf{X}_{n \times p}\) has p linearly independent columns, it may be factored in the form

                                               \(\underset{n \times p}{\mathbf{X}}=\underset{n}{\mathbf{Q}} \mathbf{R}\)

where the columns of Q are orthogonal \(\left(\mathbf{Q}^T \mathbf{Q}=\mathbf{I}\right)\) and R is upper-triangular \(\left(r_{i j}=0\right.\), for \(\left.i>j\right)\) and nonsingular. [For a discussion of this decomposition and its relationship to the Gram-Schmidt process, see Strang (1980).]

Show that \(\hat{\boldsymbol{\beta}}=\left(\mathbf{X}^T \mathbf{X}\right)^{-1} \mathbf{X}^T \mathbf{Y}\) may also be expressed as \(\hat{\boldsymbol{\beta}}=\mathbf{R}^{-1} \mathbf{Q}^T \mathbf{Y}\), or \(\mathbf{R} \hat{\boldsymbol{\beta}}=\mathbf{Q}^T \mathbf{Y}\). Indicate how this last equation may be solved for \(\hat{\boldsymbol{\beta}}\) by back substitution, using that R is upper-triangular, and show that it is thus unnecessary to invert R.