# Minimizing the sum of squared errors in linear regression (proof/matrix notation)

I'd appreciate you helping me understanding the proof of minimizing the sum of squared errors in linear regression models using matrix notation. I'm trying to derive by minimizing the sum of squared errors,

Look at this proof,

The q.c.e. basic equation in matrix form is:
y = Xb + e
where y (dependent variable) is (nx1) or  (5x1)
X (independent vars) is (nxk) or  (5x3)
b (betas) is (kx1) or  (3x1)
e (errors) is (nx1) or  (5x1)
Minimizing sum or squared errors using calculus results in the OLS eqn:
b=(X'X)-1.X'y
To minimize the sum of squared errors of
a k dimensional line that describes the relationship
between the k independent variables and y we
find the set of slopes (betas) that minimizes
Σ_{i=1 to n} e_i^2
Re-written in linear algebra we seek to min e'e
Rearranging the regression model equation, we get e = y - Xb
So e'e = (y-Xb)'(y-Xb) = y'y - 2b'X'y + b'X'Xb   (see Judge et al (1985) p14 )
Differentiating by b we get 0 = - 2X'y + X'Xb -> 2X'Xb=2X'y
Rearranging, dividing both sides by 2 -> b = X'X-1X'y


it is stated that if you rewrite the summation expression above, it is

Is any summation over a squared vector/matrix the product of the transpose with the vector/matrix itself?

Further in the proof, e is substituted by Y-X*beta, and e'e is differentiated with respect to beta, yielding this expression

and it is stated that this is equal to

Does anybody know where the second 2 comes from? If I rearrange it, it yields X'X*beta = 2X'Y without the 2 in front of the lefthand side of the equation.

Thank you!

• (1) You know how when you differentiate $x^2$ "a two comes out in front" and you get $2x$? The same happens in matrix calculus so when you differentiate b'X'Xb for b, you get 2X'Xb. They left that 2 out (typo), they should have written 0 = - 2X'y + 2X'Xb. Both twos should be there. Sep 26, 2020 at 10:02
• (2) When $x$ and $y$ are vectors $x' y$ is the inner product of $x$ and $y$. And yes, the inner product of a vector with itself (i.e. $e' e$) is equal to the sum of squares of the elements. But if $x$ or $y$ are matrices, it is not the case. Sep 26, 2020 at 10:13
• Thank you very much noob2!
– Phil
Sep 26, 2020 at 10:16