Suppose we have a set of points
\[\{(x_1,y_i)@i=1,2,...,n \}\]
and we want to find a quadratic expression \[y=a+bx+cx^2\]
to model this data.The least squares method seeks to minimise the sum of the square errors
\[E=\sum^nn_{i=1} e^2_i =\sum^n_{i=1} (y_i-(a+bx_i+cx^2_i))^2\]
.Differentiate with respect to
\[a, \: b, \: c\]
in turn, and put the results equal to zero.
\[\frac{\partial E}{\partial a}=2 \sum^n_{i=1} (y_i-(a+bx_i+cx^2_i))=0\]
\[\frac{\partial E}{\partial b}=2 \sum^n_{i=1} x_i(y_i-(a+bx_i+cx^2_i))=0\]
\[\frac{\partial E}{\partial c}=2 \sum^n_{i=1} x^2_i(y_i-(a+bx_i+cx^2_i))=0\]
From the first of these,
\[ \sum^n_{i=1} y_i=an+b \sum^n_{i=1}x_i+c \sum^n_{i=1} x^2_i\]
From the second of these,
\[ \sum^n_{i=1} y_i x_i=a \sum^n_{i=1} x_i+b \sum^n_{i=1}x^2_i+c \sum^n_{i=1} x^3_i\]
From the third of these,
\[ \sum^n_{i=1} y_i x^2_i=a \sum^n_{i=1} x^2_i+b \sum^n_{i=1}x^3_i+c \sum^n_{i=1} x^4_i\]
We can write this system in matrix form as
\[\begin{pmatrix}\sum^n_{i=1} y_i \\ \sum^n_{i=1} y_i x_i \\ \sum^n_{i=1} y_i x^2_i \end{pmatrix} = \left( \begin{array}{ccc} n & \sum^n_{i=1}x_i & \sum^n_{i=1} x^2 \\ \sum^n_{i=1} x_i & \sum^n_{i=1}x^2_i & \sum^n_{i=1} x^3_i \\ \sum^n_{i=1} x^2_i & \sum^n_{i=1}x^3_i & \sum^n_{i=1} x^4_i \end{array} \right) \begin{pmatrix}a\\b\\c\end{pmatrix}\]
.\[\begin{pmatrix}a\\b\\c\end{pmatrix} ={\left( \begin{array}{ccc} n & \sum^n_{i=1}x_i & \sum^n_{i=1} x^2_i \\ \sum^n_{i=1} x_i & \sum^n_{i=1}x^2_i & \sum^n_{i=1} x^3_i \\ \sum^n_{i=1} x^2_i & \sum^n_{i=1}x^3_i & \sum^n_{i=1} x^4_i \end{array} \right)}^{-1} \begin{pmatrix}\sum^n_{i=1} y_i \\ \sum^n_{i=1} y_i x_i \\ \sum^n_{i=1} y_i x^2_i \end{pmatrix} \]
.For the data points
\[(-5,2),(-4,7),(-3,9),(-2,12),(-1,13),(0,14),(1,14),(2,13),3,10),(4,8),(5,14)\]
.
\[\sum^n_{i=1}x_i=0, \: \sum^n_{i=1}y_i=106, \: \sum^n_{i=1}x_iy_i=20, \: \sum^n_{i=1}x^2_i=110, \: \sum^n_{i=1}x^3_i=0, \:\sum^n_{i=1}x^4_i=1958, \: \sum^n_{i=1}y_i x^2_i=688\]
.The matrix equation above becomes
\[\begin{pmatrix}a\\b\\c\end{pmatrix} ={\left( \begin{array}{ccc} 11 & 0 & 110 \\ 0 & 110 & 0 \\ 110 & 0 & 1958 \end{array} \right)}^{-1} \begin{pmatrix}106 \\ 20 \\ 688 \end{pmatrix} =\begin{pmatrix}13.9\\0.18\\-0.43\end{pmatrix} \]
(to 2 decimal places).
The model is
\[y=14.9+0.18x-0.43x^2\]
.