Linear LMS estimation based on two observations. Consider

Chapter , Problem 21

(choose chapter or problem)

Linear LMS estimation based on two observations. Consider three random variables e, X, and Y, with known variances and covariances. Assume that var(X) > 0, var(Y) > 0, and that Ip(X, Y) I =j:. 1. Give a formula for the linear LMS estimator of e based on X and Y, assuming that X and Y are uncorrelated, and also in the general case. Solu tion. We consider a linear estimator of the form e = aX + bY + c and choose a, b, and c to minimize the mean squared error E [(e - aX - bY - C) 2 ] . Suppose that a and b have already been chosen. Then, c must minimize E [(8 - aX - bY - C) 2 ] , so c = E[e] - aE[X] - bErYl . It follows that a and b minimize E [( (8 - E[8]) - a(X - E[X]) - b(Y - E[Y])) 2 ] . We may thus assume that 8, X, and Y are zero mean, and in the final formula subtract the means. Under this assumption, the mean squared error is equal to E [(8 - aX - by) 2 ] = E[82 ] + a2E[X2] + b2E[y2 ] _ 2aE[8X] - 2bE[8Y] + 2abE[XY] . Assume that X and Y are uncorrelated, so that E[XY] = E[X] E[Y] = O. We differentiate the expression for the mean squared error with respect to a and b, and set the derivatives to zero to obtain _ E[8X] _ cov(8, X) a - - , E[X2] var(X) Thus, the linear LMS estimator is b = E[8Y] E[Y2] cov(8, Y) var(Y) . 8 = E[8] + cov( 8, X) (X _ E[Xl) + cov( 8, Y) (Y - E[Y]). var(X) var( Y) If X and Y are correlated, we similarly set the derivatives of the mean squared error to zero. We obtain and then solve a system of two linear equations in the unknowns a and b, whose solution is var(Y)cov(8, X) - cov(8, Y)cov(X, Y) a -- ------ var(X)var(Y) - cov2(X, Y) , b = var(X)cov(8, Y) - cov(8, X)cov(X, Y) . var(X)var(Y) - cov2(X, Y) Note that the assumption Ip(X, Y) I =/:. 1 guarantees that the denominator in the preceding two equations is nonzero.