1
$\begingroup$

Can someone give a mathematical proof as to why including a constant in a linear regression equivalent is to running a regression with demeaned data and zero constant?

More specifically, consider the linear regression $$Y = b_0 + b_1 X_1 + b _2 X_2 + ... b_k X_k + e$$ where $X$'s and $Y$ are vectors, and the same regression with demeaned regressors $$\bar{Y} = b_1 \bar{X}_1 + b _2 \bar{X}_2 + ... b_k \bar{X}_k + e,$$ where the $\bar{Y}$ and the $\bar{X}_i$ are demeaned. It turns out that you'll obtain the same coefficients $\{b_i\}_{1 \leq i \leq k}$ in both regressions, but I struggle to prove this.

$\endgroup$
1
$\begingroup$

Including a constant is "equivalent" to working on demeaned data only for certain cases. However among these cases is ordinary least squares (OLS) regression.

Basically, if you think of your fitting procedure as following a method of moments then you will have equivalence. Take our model as $$ Y = \alpha + \beta X + u $$

Say our model coefficients were chosen such that $E(u) \neq 0$, then the presumed symmetry of the distribution of $u$ would mean a "better" model is available by adding a further constant to $\alpha$. Therefore, we have a primary statistical condition that the first moment of $u$ be zero, and therefore

$$ E(Y)=\alpha+ \beta E(X) $$

Now, there are plenty of ways to fit regressions that are not methods of moments. For example, various M-estimators are "moment-like" but do not share this equivalence.

$\endgroup$
  • $\begingroup$ Right, in OLS the error term $e$ has an expected value of 0. So if I think of the $X$ and the $Y$'s as random variables, then $E(Y) = b_0 + b_1 E(X_1) + \cdots + b_k E(X_k) + e$. Subtracting this from the original equation gives me the desired result. $\endgroup$ – elliptics Apr 26 '14 at 16:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.