Minimizing The Score

Recall from calculus that a differentiable function is minimized at the point at which the derivative is zero. Thus, we want to find

$\begin{eqnarraystar} D((Ax-b)^2) &=& 0 \\ 2 D(Ax-b)^T (Ax-b) &=& 0 \\ A^T (Ax-b) &=& 0 \\ A^T A x &=& A^T b.\end{eqnarraystar}$

Cool! It's just another system of linear equations to solve. (Thank goodness derivatives are linear operators!)

Next: Example Up: LEAST SQUARES Previous: Least Squares in Matrices