5
$\begingroup$

When learning econometrics I have usually seen stuff from the following perspective:

  1. Assume $Y_t = f(X_t) + e_t$, where f is some function of $X_t$ (typically linear). For example, assume $Y_t = X_t * \beta + e_t$. Then if $e_t$ satisfies certain properties the OLS estimator will converge to beta.

However I have also seen, but less frequently:

  1. Make no assumption on the function relationship between $Y_t$ and $X_t$. Without any assumptions we know there exists an optimal linear approximation of $E[Y_t|X_t]$ (the alpha such that $X_t*\alpha + e_t$ minimizes MSE, for example). Now if we assume that $(Y_t,X_t)$ is covariance stationary, the OLS estimator converges to alpha.

To me it seems like the perspective of 2. is more interesting because the analysis is not predicated on assuming that Y and X have a specific functional relationship. Instead, assumptions like "covariance stationary" seem more general than assuming that $Y = a + bX + e$.

Is there a reason why there seems to be more of a focus on 1.? Are the two perspectives related in some way?

Improve this question
$\endgroup$
1
  • $\begingroup$ covariance stationary is very strong assumption. it is rare when it is true. $\endgroup$
    – Wisentgenus
    Jun 16, 2015 at 19:18

2 Answers 2

Reset to default
1
$\begingroup$

Approach 1 is parametric regression, whereas approach 2 is non-parametric regression.

How are they related: non-parametric regression models the entire distribution of all possible function forms, and then do the integration to calculate a single value E[Y|X]. It is function-form free. In contrast, parametric linear regression ASSUMES that the function form can be well described by a simple linear relation between Y and X.

So, yes, approach 2 is more flexible than approach 1. However, such flexibility does come at a cost. To reach an acceptable standard error in estimates, Non-parametric regression typically requires a much much larger amount of observations than the simple linear regression. Also, linear regression has a straightforward interpretation (1 unit increase in X would drive Y up by alpha units), but non-parametric regression result is not so intuitive (you can think of it as an weighted average of Y taken around the neighbourhood of X=X0).

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thanks for the response. Is approach 2 non-parametric though? I am thinking of fitting a parametric model (linear regression) to estimate E[Y|X]. I just want to see what analysis I can make about convergence to the best linear prediction without making assumption such as E[Y|X] is a linear function of X. For example, E[Y|X] could be X + X^3 but there would still be a best linear prediction over the probability distribution of X. $\endgroup$
    – applicative_x
    Jun 17, 2015 at 21:22
  • $\begingroup$ Approach 2 is non-parametric. If you are fitting E[Y|X] = a + bX + cX^3, it still assumes a functional form (and hence can contain significant bias, for example, why cubic? why not include X^4? any theories support this specs? even worse, X and X^3 may be highly correlated, introducing multi-collinearity problem.) What's more, it is still 'linear' in parameters (a, b, c) from a statistical perspective (although it is called poly regression). $\endgroup$
    – Jianxun Li
    Jun 17, 2015 at 22:53
  • $\begingroup$ Alternatively, you can think it this way. Let's treat the functional form as a single parameter, theta. Non-parametric approach believes that this theta has a distribution to characterize its uncertainty. So, to work out the expectation, we need to integrate over all possible functional forms (poly, exp, linear, etc). In contrast, both linear regression or polynomial regression make our life easier by only consider a particular choice of functional forms. The cost is obviously that extracting full distribution info using just a single point would yield bias. $\endgroup$
    – Jianxun Li
    Jun 17, 2015 at 22:59
  • $\begingroup$ Thanks Jianxun, I will look into the non parametric literature. My thinking was that the only reason to estimate E[Y|X] as (a + bX) even if the expectation is non-linear would be to reduce overfitting/variance of the estimator. I am interested in finding out under what conditions is fitting a linear model to E[Y|X] good? For example, if (Y,X) are non-stationary and not cointegrated, the estimator variance will not converge to 0 asymptotically so it may be a bad approach. $\endgroup$
    – applicative_x
    Jun 17, 2015 at 23:18
-1
$\begingroup$

look at the econometrics literature on "total Least squares" (van huffel has a text out by that name)...or more generally think about what principal components does (hint: it's minimizing the distance to a regression line as in #2..it's not minimizing just the "vertical" distance)

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.