$\mathbb{P}$ vs $\mathbb{Q}$ Probabilities - Transitioning Between Measures

Question

I'd like this question to definitively guide a practitioner to using both $\mathbb{P}$ vs $\mathbb{Q}$ probabilities in trading and research.

Let's take only one fact as given: if I have a risk-neutral probability distribution I can price and hedge any option.

1. Is the distinction more philosophical or practical? Does it have real impact on trading desks P/L? For example, is it a construct to remind us we're not in the "real world" when modeling?
2. This question says it is the difference in using $\mu$ vs $r$ when solving the S.D.E... which seems to say if I definitively knew $\mu$ and $r$ I would be able to transition with absolutely no loss of information. What edge would this provide me in the market?
3. This good paper and this good answer seems to divide them on the approach of their research. $\mathbb{P}$-quants vs $\mathbb{Q}$-quants... in this sense it seems to be that $\mathbb{P}$-Quants are concerned with modeling the future using historical data sets. Projection. $\mathbb{Q}$-quants are concered with relative valuation and making sure that their pricing shemes are consistent with exchange traded products that are observed in the market. Extrapolation. I see that these job functions are different, but I do not see why one could not apply $\mathbb{P}$ methods to the $\mathbb{Q}$ world (their effectiveness seems less important to me - it doesn't seem like a scientific violation).
4. Girsanov's Theorem shows its possible to switch between the two. Now I know I CAN draw conclusions from each other, but the method is not clear.

Is there a way on paper to move from $\mathbb{P}$ to $\mathbb{Q}$ and vice-versa if I have a closed form-solution or a parameterized model of either $\mathbb{P}$ or $\mathbb{Q}$? If my returns under $\mathbb{Q}$ is $X \sim \mathcal{N}(r,\,\sigma^{2})\,$. From here, how can I get to $\mathbb{P}$.

I'd prefer to stay out of a model-framework completely and let all results be in general. From what I've found I believe the connection is in putting a price on the market risk premium, but I have not found empirical estimations of this or attempts to use its estimation for moving between $\mathbb{P}$ and $\mathbb{Q}$. Any papers on $\lambda$ estimation or extraction would be appreciated.

I wanted to add this quote from Gary Hatfield:

Recall that the whole point of risk neutral pricing is to recover the price of traded options in a way that avoids arbitrage. As such, the probabilities of various paths are implied from the prices of various traded securities whose payoffs depend on those paths. Since investors are in aggregate risk averse, these prices imply higher probabilities to bad scenarios than they do to good scenarios. Hence, while everyone (almost!) agrees that stocks have a higher expected return than risk free bonds, the prices of stock and stock options imply the only difference between stocks and risk free bonds is that stocks are more volatile. Put another way, a risk neutral scenario set has many more really bad scenarios than a real world scenario set precisely because investors fear these scenarios. They therefore overweigh their probability when deciding how much a security is worth.

This provides intuitive context to the difference, but makes it seem impossible to every replicate the $\mathbb{P}$ world.

Answer 1

$\mathbb{P}$ is the true probability measure. Measure $\mathbb{Q}$ is a measure of convenience that allows risk neutral pricing. Stochastic discount factor $M$ takes you between the two.

• If you care about prices you can either: (1) work under $\mathbb{Q}$ or (2) work under $\mathbb{P}$ with a stochastic discount factor $M$.

• If you care about the real world probabilities of various outcomes, you need to use $\mathbb{P}$. If you want real world, expected returns you want $\mathbb{P}$. (In efficient markets, the expected return under $\mathbb{Q}$ of every security is the risk free rate.)

Theoretical justification behind the existence of a measure $\mathbb{Q}$

Perhaps it's useful to backup a moment and revisit basic foundations of asset pricing theory.

• Let $X$ be a random variable denoting a risky cash flow next period.
• Let $f(X)$ be a pricing function which gives today's price for a risky cash flow.

An obviously desirable feature of $f$ is that it should be linear: $f(aX + bY) = af(X) + bf(Y)$ (where $a$ and $b$ are scalars and $Y$ is another risky cash flow). If $f$ is a linear functional then there exists a stochastic discount factor $M$ such that:

$$ f(X) = \operatorname{E}^\mathbb{P}[MX]$$

If we're in a discrete probability space with $n$ possible outcomes (for simplicity), then for any linear functional $f$, there exists an $M$ such that $f$ can be written as

$$ f(X) = \sum_i p_i m_i x_i $$

That whole $M$ business may be rather inconvenient. Is there something we could do? Let $q_i = \frac{p_i m_i}{\sum_j p_j m_j}$. Observe that $\sum_i q_i = 1$ so that $Q$ is a probability measure. Also observe that the risk free rate satisfies $1 = \sum_i p_i m_i (1 + r_f)$ hence $ \sum_i p_i m_i = \frac{1}{1 + r_f}$

Then: $$ \operatorname{E}^\mathbb{P}[MX] = \sum_i p_i m_i x_i = \frac{1}{1+r_f} \sum_i q_i x_i = \frac{1}{1+r_f} \operatorname{E}^\mathbb{Q}[X]$$

For pricing, you can either use:

1. probability measure $\mathbb{P}$ along with the stochastic discount factor $M$
2. a tilted probability measure $\mathbb{Q}$ and risk neutral pricing.

For pricing purposes, they're the same thing.

Discussion

A fundamental idea of asset pricing theory is that each state of the world $i$ may have a different state price given by $p_i m_i$. The idea of risk neutral pricing is that for convenience, you can pack this into a tilted probability measure $Q$ where $q_i$ is proportional to $p_i m_i$. What takes you back and forth between $P$ and $Q$ is the stochastic discount factor $M$ aka state price deflator aka state price density aka pricing kernel.

So what might $M$ be? (economics interpretation)

Describing $M$ is the great goal of academic asset pricing. Success has been at best mixed.

With multiple time periods, $M$ is a stochastic process rather than a random variable.

Economics often calls $M$ a marginal rate of substitution process because in economic theory, if agents have additively separable utility over a consumption stream given by $U(C) = \sum_{t=0} \beta^t u(C_t)$ then the SDF is a ratio of marginal utilities:

$$ M_{t+1} = \beta \frac{u'(C_{t+1})}{u'(C_t)}$$