# Why (most) quants think that the risk-neutral measure should not be used for financial forecasting?

In posts regarding the $$\mathbb{P}$$ vs $$\mathbb{Q}$$ debate (see 1, 2, 3 or 4), most answers seem to conclude that historical-based methods are better suited than risk-neutral models for financial predictions.

However, this conclusion seems to be based more on subjetive beliefs than in empirical analyses. In fact, existing analyses comparing historical and risk-neutral predictions tend to conclude that risk-neutral models perform better than historical models in financial predictions. See, for instance, these recent papers: 1, 2 or these comprehensive surveys: 1 and 2.

While it is undeniable that risk-neutral forecasts have theoretical flaws, a rational analysis would entail: (i) evaluating how these flaws actually impact future predictions and (ii) comparing such impact with the biases observed in historical-based models. However, since most people already seem to have a notably strong view, this pre-conditioning could be relegating to second place what should be, conceptually, the core of the discussion (i.e.: performing empirical analyses and statistically comparing forecasting models with an agnostic view)

So, why it is that we adopt such pre-conditioned view regarding risk-neutral predictions?

In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of Quantitative Finance.

The topic of P vs Q is presented in that dialogue in a manner meant to be understandable to a new comer. I would certainly recommend you to have a look!

This dialogue is also available on the arXiv.

I am also adding a couple quotes from that dialogue:

.. "Q: Ok. We simulate under P because we want the risk statistics of the portfolio in the real world, under the physical probability measure, and not under the so called pricing measure Q" ..

.. "Q: And how is this Q related to P? [still puzzled] A: The two measures are related by a mathematical relationship that de- pends on risk aversion, or market price of risk. In the simplest models the real expected rate of return is given by the risk free rate plus the market price of risk times the volatility. Indeed the ”expected” return of an asset depends on the probability measure that is used. For exam- ple, under P the average rate of return of an asset is hard to estimate, whereas under Q one knows that the rate of return will be the risk free rate, since dependence on the real rate of return can be hedged away through replication techniques. [starts looking tired]"

There is a deeper issue. Frequentist distributions are not probability distributions because they are designed to be minimax distributions rather than actual distributions. This ignores all of the other problems and this also ignores risk-neutral versus any other measure of risk aversion.

An even deeper issue is that these models presume that the parameters are known. If you have a wealth model such as $w_{t+1}=Rw_t+\epsilon_{t+1}$, where $\epsilon_t\sim\mathcal{N}(0,\sigma^2_t)$, then if $R$ and $\sigma^2_t$ are known, then the mean-variance results follow. However, if the parameters are not known, then they can never be known.

See:

White, J. S. (1958). The limiting distribution of the serial correlation coefficient in the explosive case. The Annals of Mathematical Statistics, 29(4):1188–1197

This shocking proof, when combined with Mann and Wald's proof

Mann, H. and Wald, A. (1943). On the statistical treatment of linear stochastic difference equations. Econometrica, 11:173–200.

Imply that if mean-variance models are true, then no solution can be constructed using their axioms.

For a solution to the question you are asking, however, see

Harris, D.E.(2017) The Distribution of Returns. Journal of Mathematical Finance, 7, 769-804,

which derives the distributions of returns. You can use these as Bayesian likelihood functions and arrive at a coherent predictive distribution.

The Bayesian predictive distribution is: $$\Pr(\tilde{x}=k|\mathbf{X})=\int_{\theta\in\Theta}\Pr(\tilde{x}=k|\theta)\Pr(\theta|\mathbf{X})\mathrm{d}\theta,\forall{k}\in\chi,$$ where $\chi$ is the sample space, $\Theta$ is the parameter space, $\theta$ is a parameter or vector of parameters in $\Theta$, $\mathbf{X}$ is the observed sample and $\tilde{x}$ is some predicted value.

Notice that $\Pr(\tilde{x}|\mathbf{X})$ does not depend upon the values for $\theta$ as any uncertainty as to its true value has been marginalized out so that the prediction only depends upon the information that you observed.

The predictive distribution of $\tilde{x}$ is coherent so that a bookie, or market-maker, offering prices or odds based on the information in $\mathbf{X}$ cannot be gamed and take a sure loss by a clever actor or set of actors. That is never true for Frequentist solutions. You can rig a game based on a Frequentist statistic if you are crafty enough.

Whereas the Frequentist model assumes the parameters are known, the Bayesian model ends with a result that they do not matter for the purposes of making a prediction. That is a powerful result. The former is fragile, the latter is robust.

As to why we adopt these strong views, it is because we are people. Even economists who do not work for the Street have vested interests. It is challenging to be a scientist when your salary is tied to your outcomes.

Perhaps a case of views based upon theoretical possibilities rather than empirical realities?

### In theory, $P$ and $Q$ can be extremely different

• $P$ is the real world, actual probability measure.
• $Q$ is an artificial probability measure constructed so that security prices are risk neutral expectations (discounted at risk free rate) under measure $Q$.

The connection between the two is through state prices. If state prices are constant, then $P$ and $Q$ are the same. But with arbitrary state prices, $P$ and $Q$ can be made arbitrarily different from each other. The price of hurricane insurance and the risk neutral probability of a hurricane can deviate arbitrarily from the probability of a hurricane depending on the market price of hurricane risk. If risk prices are extreme enough, a 1% probability event may have 20% probability under the risk neutral measure. (Economic theory may have some objection to this, but the basic math of risk neutral pricing does not.)

### Quick review: the pricing kernel connects $P$ and $Q$

The linearity of a linear asset pricing function $p(X)$ that prices a payoff $X$ implies that it can be written as an inner product with a pricing kernel. $$p(X) = \mathbb{E}^P\left[ \pi X \right]$$

Then define Radon-Nikodym derivative $\frac{dQ}{dP} = \pi R_f$ and conduct a change of measure. $$p(X) = \frac{1}{R_f}\mathbb{E}^P\left[\frac{dQ}{dP}X \right] = \frac{1}{R_f} \mathbb{E}^Q[X]$$

So depending what pricing kernel $\pi$ is, some notion of distance between $P$ and $Q$ could be quite large (since the linear $p$ assumption on its own doesn't put meaningful constraints on $\pi$).

The risk neutral density is a mathematical trick to allow pricing of options. As it has little bearing on reality, it makes little sense to simulate from it for the purposes of forecasting real results.

I would argue that any paper that shows that the risk neutral density forecasts better than the real world is either suffering from a specification or an estimation problem.

Perhaps the best empirical study that I have seen on the topic is the following paper, which examines the real world density, the risk neutral density, and the link between the two for the (very general) CGMY model: