Fama Macbeth regression and portfolio sort result contradiction

Question

I ran Fama Macbeth (regression) on two variables called return and lag MAX ( monthly average return and lag of maximum return over a month). the results are like the following :

           Estimate Std. Error t value  Pr(>|t|)    
(Intercept) 0.018100   0.004911  3.6856 0.0002281 ***
MAX_1       0.030758   0.034241  0.8983 0.3690355 

but the portfolios sort result by MAX_1 variable is like this

MAX sort return
  low MAX_1 0.675
    port2   0.760
    port3   1.920
    port4   1.538
    port5   1.974
    port6   2.154
    port7   2.543
    port8   2.548
    port9   2.949
 High MAX_1 4.506
  diff 10-1 3.832***
    t      (3.231)

As you see that the return difference of two extreme portfolios is positive and significant. However, the MAX_1 coefficient is positive but not significant. Could anyone can explain that why Fama Macbeth regression is not significant? for your convenience, I am giving the summary stat of lag MAX

        vars    mean    sd    median  trimmed   mad      min       max   range  
max_1   7.000   0.072   0.444   0.050   0.059   0.031   -0.957  180.080  181.037    

Answer 1

There are a couple of issues here, none of which are trivial.

First, p-values are not transitive. Consider the data set: $A=\{1,2,3\}$, $B=\{3,4,5\}$, and $\{5,6,7\}.$ Pairwise, the differences between A and B, and B and C are not statistically significant, but C-A is. There is no reason for the grand mean to be significant, even if the extreme differences are significant.

Second, the null hypothesis is highly informative. The assertion in the null is that there is a one hundred percent chance that the null is true. That is a very strong statement. A p-value is the probability of seeing a result as extreme or more extreme conditioned on the truth of the null hypothesis. It is $\Pr(X|\theta=k)$, where $X$ is the data and $\theta$ is some parameter.

Consider the results in:

Wetzels, R., Matzke, D., Lee, M. D., Rouder, J. N., Iverson, G. J., & Wagenmakers, E.-J. (2011). Statistical Evidence in Experimental Psychology: An Empirical Comparison Using 855 t Tests. Perspectives on Psychological Science, 6(3), 291–298. https://doi.org/10.1177/1745691611406923

They rechecked 855 t-tests appearing in the literature by converting them to Bayesian probability statements. Many statistically significant results under the Frequentist hypothesis testing regime came out non-significant. Some came out as supporting the null. Bayesian methods do not test to see if a hypothesis is false, it tests to see if they are true. In many cases, the null was more probably true than the alternative, but the Frequentist result said the opposite.

While the results of the two methods showed a high degree of concordance, as would be expected, the strong assertion that the null is true is information and appears to be the source of the differences where they happen.

Finally, if you used raw data rather than log data, then your data has no first or second moment. Look at your mean and your range! That is probably not an outlier. See the proof in https://economics.stackexchange.com/questions/26033/log-returns-in-fianance.

If the data is approximately a truncated Cauchy distribution, then $\beta$ as understood in Fama-MacBeth does not exist. It is unclear, however, what it means in logarithmic form. In logarithmic form, the likelihood function is the hyperbolic secant distribution, but that distribution has no covariance matrix. The central limit theorem holds, so least squares regression is usable, but it is unclear what it means. In the raw form of the data, a coscale parameter exists, but collapses into the joint relationship as a determinant. The result is that no variables can covary, but none are independent either. Rather, a copula relationship holds.

A standardized package is not built to solve this problem because neither likelihood, the hyperbolic secant or the Cauchy, has a sufficient statistic. The inferences in the log case should be valid because the pivotal quantity is normally distributed, so distortions in $\hat{\beta}$ are matched by the distortions in the standard error. Still, you wouldn't want to build an actual portfolio on it. The log transformation overstates return by two percent over the raw value and understates risk by four percent. I did a population study of annual returns in the CRSP universe. That was the result.

Your range is wide because the fifty percent interval for the Cauchy distribution is $\pm{1}\sigma$, but the 99.95% interval is $\pm{636}\sigma$ hence your wide range.

First remember that the only goal of Fama-MacBeth was to falsify a the validity of the CAPM, not to create a new model. They succeeded in falsifying it, but that does not imply that their model is valid. Everyone is using as if it is valid because of Frequentist decision theory.

If their model has been a Fisherian model using the method of maximum likelihood, then their model would only have falsified the null. Once the null is false, you know it isn't true so you go on to other things. However, under Pearson and Neyman's Frequentist decision theory, you are to behave as if the alternative is true. This is because it creates a binary choice, either the CAPM or Fama-MacBeth. Since you are in the rejection area for the CAPM, then you accept Fama-MacBeth as if true.

One note, to properly use Fama-MacBeth, you must first falsify the CAPM. Fama-MacBeth is not the default model in that decision-theoretic construction, the CAPM is!