# Fama Macbeth regression and portfolio sort result contradiction

I ran Fama Macbeth (regression) on two variables called return and lag MAX ( monthly average return and lag of maximum return over a month). the results are like the following :

           Estimate Std. Error t value  Pr(>|t|)
(Intercept) 0.018100   0.004911  3.6856 0.0002281 ***
MAX_1       0.030758   0.034241  0.8983 0.3690355


but the portfolios sort result by MAX_1 variable is like this

MAX sort return
low MAX_1 0.675
port2   0.760
port3   1.920
port4   1.538
port5   1.974
port6   2.154
port7   2.543
port8   2.548
port9   2.949
High MAX_1 4.506
diff 10-1 3.832***
t      (3.231)


As you see that the return difference of two extreme portfolios is positive and significant. However, the MAX_1 coefficient is positive but not significant. Could anyone can explain that why Fama Macbeth regression is not significant? for your convenience, I am giving the summary stat of lag MAX

        vars    mean    sd    median  trimmed   mad      min       max   range
max_1   7.000   0.072   0.444   0.050   0.059   0.031   -0.957  180.080  181.037

• What capital market and time horizons are you looking at? Typically, the MAX-effect is strongly negative: Stocks with high MAX, i.e. with an observation of a high daily return during the previous month, have a low (monthly) return in the subsequent month, see e.g. Bali et al (2011). It would be helpful for giving an answer if you would provide some summary statistics on your variables (especially on MAX). – skoestlmeier Jan 16 '19 at 16:35
• I know most of the countries have negative max. but there is some exception like Canada. you can check this one sciencedirect.com/science/article/abs/pii/S0927539816300196 – Max Jan 17 '19 at 7:43
• I have edited and add summary stat @skoestlmeier – Max Jan 17 '19 at 8:04
• Did you exclude certain stocks, like financial stocks or illiquid stocks (e.g. <5% cross-sectional market valuation), from your sample prior to the analysis? A maximum of max of 181,04% seems like you do not winsorize data etc. In comparison to portfolio sorts, any regression as a parametric approach is very sensitive to any outliers/extreme values. – skoestlmeier Jan 17 '19 at 8:35

There are a couple of issues here, none of which are trivial.

First, p-values are not transitive. Consider the data set: $$A=\{1,2,3\}$$, $$B=\{3,4,5\}$$, and $$\{5,6,7\}.$$ Pairwise, the differences between A and B, and B and C are not statistically significant, but C-A is. There is no reason for the grand mean to be significant, even if the extreme differences are significant.

Second, the null hypothesis is highly informative. The assertion in the null is that there is a one hundred percent chance that the null is true. That is a very strong statement. A p-value is the probability of seeing a result as extreme or more extreme conditioned on the truth of the null hypothesis. It is $$\Pr(X|\theta=k)$$, where $$X$$ is the data and $$\theta$$ is some parameter.

Consider the results in:

Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011). Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests. Perspectives on Psychological Science, 6(3), 291–298. https://doi.org/10.1177/1745691611406923

They rechecked 855 t-tests appearing in the literature by converting them to Bayesian probability statements. Many statistically significant results under the Frequentist hypothesis testing regime came out non-significant. Some came out as supporting the null. Bayesian methods do not test to see if a hypothesis is false, it tests to see if they are true. In many cases, the null was more probably true than the alternative, but the Frequentist result said the opposite.

While the results of the two methods showed a high degree of concordance, as would be expected, the strong assertion that the null is true is information and appears to be the source of the differences where they happen.

Finally, if you used raw data rather than log data, then your data has no first or second moment. Look at your mean and your range! That is probably not an outlier. See the proof in https://economics.stackexchange.com/questions/26033/log-returns-in-fianance.

If the data is approximately a truncated Cauchy distribution, then $$\beta$$ as understood in Fama-MacBeth does not exist. It is unclear, however, what it means in logarithmic form. In logarithmic form, the likelihood function is the hyperbolic secant distribution, but that distribution has no covariance matrix. The central limit theorem holds, so least squares regression is usable, but it is unclear what it means. In the raw form of the data, a coscale parameter exists, but collapses into the joint relationship as a determinant. The result is that no variables can covary, but none are independent either. Rather, a copula relationship holds.

A standardized package is not built to solve this problem because neither likelihood, the hyperbolic secant or the Cauchy, has a sufficient statistic. The inferences in the log case should be valid because the pivotal quantity is normally distributed, so distortions in $$\hat{\beta}$$ are matched by the distortions in the standard error. Still, you wouldn't want to build an actual portfolio on it. The log transformation overstates return by two percent over the raw value and understates risk by four percent. I did a population study of annual returns in the CRSP universe. That was the result.

Your range is wide because the fifty percent interval for the Cauchy distribution is $$\pm{1}\sigma$$, but the 99.95% interval is $$\pm{636}\sigma$$ hence your wide range.

First remember that the only goal of Fama-MacBeth was to falsify a the validity of the CAPM, not to create a new model. They succeeded in falsifying it, but that does not imply that their model is valid. Everyone is using as if it is valid because of Frequentist decision theory.

If their model has been a Fisherian model using the method of maximum likelihood, then their model would only have falsified the null. Once the null is false, you know it isn't true so you go on to other things. However, under Pearson and Neyman's Frequentist decision theory, you are to behave as if the alternative is true. This is because it creates a binary choice, either the CAPM or Fama-MacBeth. Since you are in the rejection area for the CAPM, then you accept Fama-MacBeth as if true.

One note, to properly use Fama-MacBeth, you must first falsify the CAPM. Fama-MacBeth is not the default model in that decision-theoretic construction, the CAPM is!

• May you please explain why you refer Fama/MacBeth to be a model? In my view, it is just a two-step regression method to retrieve robust stand. errors to correct for cross-sect. correlation. It was first used to test the CAPM, but is applicable in general to test any cs-relationship. Fama(2008) showed that Fama/MacBeth and portfolio sorts are in general redundant and different results indicate the effect being rooted in some subportfios (e.g. microcaps). So port. sorts capture these nonlinear relations whereas regressions as parametric methods fail to so (which could explain the above results). – skoestlmeier Jun 21 '20 at 11:54
• I was speaking strictly within the results of decision theory, not economic model building. Frequentist inference and Frequentist decisions are inseparable. If you stop at inference and make no decision regarding it, then it is not a model. It is just an observation that the CAPM is falsified empirically. Of course, if you are managing money, then you are making decisions. The alternative becomes the model as it is a binary, forced-choice. That is not true under Fisher's logic. It also isn't automatically true, if you stop at inference. – Dave Harris Jun 22 '20 at 1:11