Derivation of Expressions for Multiple Regression Coefficients

Supose we suspect that an observable  
  and random variables  
\[X+- , \: i=1,2,3,...n\]
  are linearly related. We can try to find an relationship of the form  
\[y_i =\beta_0+\beta_1(x_{i1}- \bar{x}_1)+ \beta_2(x_{i2}-\bar{x}_2))+...+\beta_n(x_{in}- \bar{x}_n)\]
The statistically most desirable way to find the coefficients  
\[\beta_k, \: k=1,2,...,n\]
  is to minimise the sum of the squares of the errors, so that if the observable  
  and the random variables  
\[X_i, \: 1,2,3,...,n\]
  to have paired values, then we want to minimise  
\[E=\sum^n_{i=1} e^2_i= \sum^n_{i=1} (y_i-(\beta_0+\beta_1(x_{i1}- \bar{x}_1)+ \beta_2(x_{i2}-\bar{x}_2))+...+\beta_n(x_{in}- \bar{x}_n))^2 \]
Find the partial derivatives and set each equal to zero.
\[\frac{\partial E}{\partial \beta_0}=\sum^n_{i=1} 2 (\beta_0 - y_i) =0 \rightarrow n \beta_0 = \sum^n_{i=1} y_i \rightarrow \beta_0 = \frac{\sum^n_{i=1} y_i}{n} \]

\[\frac{\partial E}{\partial \beta_k}=\sum^n_{i=1} (-2(x_{ik}-\bar{x}_k) (y_i- \bar{y})+ 2(x_{ik}- \bar{x}_k)^2 =0 \rightarrow n \beta_k = \frac{\sum^n_{i=1} (x_{ik}-\bar{x}_k) (y_i- \bar{y})}{\sum^n_{i=1} (x_{ik}- \bar{x}_k)^2}\]

Add comment

Security code