4950 A common problem in experimental work is to obtaina mathematical relationship y =

Chapter 13, Problem 49

(choose chapter or problem)

A common problem in experimental work is to obtain a mathematical relationship \(y=f(x)\) between two variables \(x\) and \(y\) by “fitting” a curve to points in the plane that correspond to experimentally determined values of \(x\) and \(y\), say

                   

                                                         \(\left(x_{1}, y_{1}\right),\left(x_{2}, y_{2}\right), \ldots,\left(x_{n}, y_{n}\right)\)

The curve \(y=f(x)\) is called a mathematical model of the data. The general form of the function \(f\) is commonly determined by some underlying physical principle, but sometimes it is just determined by the pattern of the data. We are concerned with fitting a straight line \(y=mx+b\) to data. Usually, the data will not lie on a line (possibly due to experimental error or variations in experimental conditions), so the problem is to find a line that fits the data “best” according to some criterion. One criterion for selecting the line of best fit is to choose \(m\) and \(b\) to minimize the function

         

                                                   \(g(m, b)=\sum_{i=1}^{n}\left(m x_{i}+b-y_{i}\right)^{2}\)                                  

This is called the method of least squares, and the resulting line is called the regression line or the least squares line of best fit. Geometrically, \(\left|m x_{i}+b-y_{i}\right|\) is the vertical distance between the data point \(\left(x_{i}, y_{i}\right)\) and the line \(y=mx+b\).

                                   

These vertical distances are called the residuals of the data points, so the effect of minimizing \(g(m,b)\) is to minimize the sum of the squares of the residuals. In these exercises, we will derive a formula for the regression line.

The purpose of this exercise is to find the values of \(m\) and \(b\) that produce the regression line.

(a) To minimize \(g(m,b)\), we start by finding values of \(m\) and \(b\) such that \(\partial g / \partial m=0\) and \(\partial g / \partial b=0\). Show that these equations are satisfied if \(m\) and \(b\) satisfy the conditions

   

   

                                           \(\left(\sum_{i=1}^{n} x_{i}^{2}\right) m+\left(\sum_{i=1}^{n} x_{i}\right) b=\sum_{i=1}^{n} x_{i} y_{i}\)

                                        \(\left(\sum_{i=1}^{n} x_{i}\right) m+n b \sum_{i=1}^{n} y_{i}\)

Let \(\bar{x}=\left(x_{1}+x_{2}+\cdots+x_{n}\right) / n\) denote the arithmetic average of \(x_{1}, x_{2}, \ldots, x_{n}\). Use the fact that

                                                 \(\sum_{i=1}^{n}\left(x_{i}-\bar{x}\right)^{2} \geq 0\)

 

to show that

                                        \(n\left(\sum_{i=1}^{n} x_{i}^{2}\right)-\left(\sum_{i=1}^{n} x_{i}\right)^{2} \geq 0\)

with equality if and only if all the \(x_{i}\) ’s are the same.

(c) Assuming that not all the \(x_{i}\) ’s are the same, prove that the equations in part (a) have the unique solution

                                   \(m=\frac{n \sum_{i=1}^{n} x_{i} y_{i}-\sum_{i=1}^{n} x_{i} \sum_{i=1}^{n} y_{i}}{n \sum_{i=1}^{n} x_{i}^{2}-\left(\sum_{i=1}^{n} x_{i}\right)^{2}}\)

                                      \(b=\frac{1}{n}\left(\sum_{i=1}^{n} y_{i}-m \sum_{i=1}^{n} x_{i}\right)\)

[Note: We have shown that \(g\) has a critical point at these values of \(m\) and \(b\). In the next exercise we will show that \(g\) has an absolute minimum at this critical point. Accepting this to be so, we have shown that the line \(y=mx+b\) is the regression line for these values of \(m\) and \(b\).]

Equation Transcription:

     

Text Transcription:

y=f(x)

x

x

y

(x_1,y_1),(x_2,y_2),...,(x_n,y_n)

y=f(x)

f

y=mx+b

m

b

g(m,b)=Sum over i=1 ^n(mx_i+b-y_i)^2

|mx_i+b-y_i|

(x_i,y_i)

y=mx +b

g(m,b)

m

b

g(m,b)

m

b