I'd appreciate you helping me understanding the proof of minimizing the sum of squared errors in linear regression models using matrix notation. I'm trying to derive by minimizing the sum of squared errors,
Look at this proof,
The q.c.e. basic equation in matrix form is:
y = Xb + e
where y (dependent variable) is (nx1) or (5x1)
X (independent vars) is (nxk) or (5x3)
b (betas) is (kx1) or (3x1)
e (errors) is (nx1) or (5x1)
Minimizing sum or squared errors using calculus results in the OLS eqn:
b=(X'X)-1.X'y
To minimize the sum of squared errors of
a k dimensional line that describes the relationship
between the k independent variables and y we
find the set of slopes (betas) that minimizes
Σ_{i=1 to n} e_i^2
Re-written in linear algebra we seek to min e'e
Rearranging the regression model equation, we get e = y - Xb
So e'e = (y-Xb)'(y-Xb) = y'y - 2b'X'y + b'X'Xb (see Judge et al (1985) p14 )
Differentiating by b we get 0 = - 2X'y + X'Xb -> 2X'Xb=2X'y
Rearranging, dividing both sides by 2 -> b = X'X-1X'y
it is stated that if you rewrite the summation expression above, it is
Is any summation over a squared vector/matrix the product of the transpose with the vector/matrix itself?
Further in the proof, e is substituted by Y-X*beta, and e'e is differentiated with respect to beta, yielding this expression
and it is stated that this is equal to
Does anybody know where the second 2 comes from? If I rearrange it, it yields X'X*beta = 2X'Y without the 2 in front of the lefthand side of the equation.
Thank you!