Testing as in Fama & MacBeth vs. comparing models as in Cochrane's lecture notes

Question

Testing a model against its extension as in Fama & MacBeth (1973)

Fama & MacBeth (1973) tested the CAPM against an alternative that the dependence between the expected excess return $E(r_{i,t}^∗)$ and the relative systematic risk $\beta_𝑖$ is nonlinear (namely, quadratic) and that the idiosyncratic risk $\sigma_i$ commands nonzero expected return. The corresponding cross-sectional regression is (using different notation from the original, but keeping the equation number for reference) $$ r_{i,t}^∗ = \lambda_0 + \lambda_1 \beta_i + \lambda_2 \beta_i^2 + \lambda_3 \sigma_i + u_{i,t}. \tag{7} $$ Since $\beta_i$ and $\sigma_i$ are not observed, a two-stage procedure is used where $\beta_i$ and $\sigma_i$ are first estimated from time-series regressions and then their fitted values are used in cross-sectional regressions (one for every time period) of the type specified above. The CAPM implies $\lambda_0=\lambda_2=\lambda_3=0$ and $\lambda_1>0$.

Let us simplify and drop the term $\lambda_2 \beta_i^2$ to obtain $$ r_{i,t}^∗ = \lambda_0 + \lambda_1 \beta_i + \lambda_3 \sigma_i + u_{i,t}. \tag{*} $$ The CAPM implies $\lambda_0=\lambda_3=0$ and $\lambda_1>0$. This can be used for testing the model. If the estimated values are statistically distinguishable from what the model implies, we have evidence against the model.

Comparing a model against a submodel as suggested by Cochrane

We could also consider $(*)$ to be a competitor of the CAPM and ask, which one is the better model? Here is what John Cochrane writes about testing one model versus another (section 14.6 in his lectures notes for the course Business 35150 Advanced Investments, p. 239-240):

1. Example. FF3F. $$ E(R^{ei}) = \alpha_i + b_i\lambda_{rmrf} + h_i\lambda_{hml} + s_i\lambda_{smb} \tag{i} $$ Do we really need the size factor? Or can we write $$ E(R^{ei}) = \alpha_i + b_i\lambda_{rmrf} + h_i\lambda_{hml} \tag{ii} $$ and do as well? ($\alpha$ will rise, but will they rise “much”?)

2. A common misconception: Measure $\lambda_{smb} = E(smb)$. If $\lambda_{smb} = 0$ (and “small”) we can drop it. Why is this wrong? Because if you drop $smb$ from the regression, $b_i$ and $h_i$ also change!

<...>

4. Solution: (a) “ First run a regression of $smb_t$ on $rmrf_t$ and $hml_t$ and take the residual, $$ smb_t = \alpha_{smb} + b_s rmrf_t + h_s hml_t + \varepsilon_t \tag{iii} $$ Now, we can drop $smb$ from the three factor model if and only $\alpha_{smb}$ is zero. Intuitively, if the other assets are enough to price $smb$, then they are enough to price anything that $smb$ prices.

Contradiction?

How do I reconcile the Fama & MacBeth approach with Cochrane's advice? Cochrane's second point is directly against what Fama & MacBeth suggest doing. Is there a contradiction? Fama & MacBeth are looking for violations of the CAPM, while Cochrane shows us how to compare alternative models. These are not entirely the same thing, but they seem to be closely related, especially since both deal with a model and a submodel (a restricted model). If I find that model B is better than model A, does it not suggest that A is violated?

References

• Cochrane, J. H. (2014). Week 5 Empirical methods notes. Business 35150 Advanced Investments, 225-247.
• Fama, E. F., & MacBeth, J. D. (1973). Risk, return, and equilibrium: Empirical tests. Journal of Political Economy, 81(3), 607-636.

Answer 1

Both Fama & Macbeth and Cochrane are playing a bit loose, statistically, in their recommended procedures here. First off, numerical noise more or less guarantees that you will measure e.g. $\lambda_0 \neq 0$ and $\alpha_{smb} \neq 0$. Without qualifying what error bars they truly mean to employ in these criteria, the question of whether they contradict is mathematically undecideable.

I will say that Cochrane's second point is no contradiction to F&M -- here Cochrane is simply noting that a univariate regression is no way to learn the model parameters.

For readers wondering what approach should be taken to distinguish between a model and a submodel, I suggest the 4-page paper Estimating the Dimension of a Model by Schwarz, in which we see that the Bayesian Information Criterion (BIC) is preferable to Akaike Information Criterion (AIC) when choosing among submodels, because the latter is not consistent.

By not recommending BIC in 2014, Cochrane is being a little sloppy. (I can't really fault a 1973 paper for lacking AIC or BIC).

Answer 2

In this answer, the terms intercept, slope coefficient and $R^2$ refer to the true parameter values corresponding to the regression model in population. They do not refer to estimates unless explicitly specified.

Comparing a model against a submodel as suggested by Cochrane

The intercept in each of the equations $(7)$, $(*)$, $(\text{i})$ and $(\text{ii})$ is informative of the validity of the corresponding asset pricing model. The model implies the intercept must be zero. If the intercept is nonzero, the model is invalid (as in at odds with the data). Adding or removing a (potential) pricing factor will affect the validity of the model unless the intercept is unchanged. The intercept in $(\text{i})$ is unchanged only if the intercept in $(\text{iii})$ is zero. This is why Cochrane recommends testing whether the intercept in $(\text{iii})$ is zero as a criterion of adding or removing a (potential) pricing factor. He cares about the validity of the model, thus the choice.

Testing a model against its extension as in Fama & MacBeth (1973)

The slope coefficient (such as $s_i$ in $(\text{i})$) corresponding to an asset's sensitivity to a (potential) pricing factor is informative of the model's explanatory power. If the asset returns vary with their factor sensitivity* conditional on sensitivities to the other factors in the model, the slope coefficient will be nonzero (and the $R^2$ will be greater in a model containing the factor than in one without it). If we are interested in whether the presence of a (potential) pricing factor adds some explanatory power to the model, we can test whether the corresponding slope coefficient equals zero.
We should keep in mind that having this additional explanatory power might come at an expense of making the pricing model invalid (or bringing it further away from validity) through its impact on the intercept. However, it is also possible that presence of the factor not only improves the model's explanatory power but also makes the model valid (or brings it closer to validity) at the same time.

When to use which approach

As Cochrane notes in points 5. and 6. on p. 240-241 of the linked lecture note, having more explanatory power is beneficial, as that makes the regression errors smaller and thus measurements better (estimates of the parameters more precise, hypothesis tests more powerful). He concludes that inclusion of a factor depends on the intended use of the model. E.g. if you want to arbitrage, including a factor with high additional explanatory power is helpful. Meanwhile, if you want a valid (or closer to valid) asset pricing model, pay attention to the effect that a factor's inclusion or exclusion has on the intercept.

*Except for the case of $\sigma_i$ in $(7)$ and $(*)$; there, $\sigma_i$ is an individual characteristic rather than factor sensitivity.