# How to estimate real-world probabilities

In the world of finance, Risk-neutral pricing allow us to estimate the fair value of derivatives using the risk free rate as the expected return of the underlyings.

However, the behavior of financial assets in the real-world might be substantially different to the evolution used in a risk-neutral context.

For instance, if I want to estimate the real-world probability of an equity asset reaching certain thresholds, which models and calibration techniques could be used?

In particular, some questions that may arise in the estimation of real-world probabilities are:

• Calibration: Should real-world probabilities be calibrated to current market prices or, alternative, historical data should be used for this type of estimation?
• No-arbitrage conditions: Could they be relaxed or they still play a role in the assessment of real-world probabilities?
• Expected returns: Assuming that I have already estimated the expected return of an asset $\mu$, how accurate would be a real-world estimation that combines a widely used evolution model (e.g. Geometric Brownian motion), with the use of $\mu$ instead of the risk free rate $r$?

Per comments, I understand that in order to estimate real-world probabilities:

• I should use expected returns instead of the risk-free rate.
• The asset evolution should still respect the no-arbitrage conditions (i.e: the real-world dynamics should still reproduce the current prices of vanilla options).

However, if we just use $\mu$ instead of $r$, the underlying asset behavior might not be consistent with the observed option prices. For instance, if we just change $r$ by $\mu$ (with $\mu>r$) the underlying asset dynamics will lead to call prices above its current market price, and put prices below its market price.

Therefore, in addition to use expected returns, which other adjustment might be needed in order to estimate real-world probabilities?

Any papers or references regarding real-world estimation will be greatly appreciated.

-
If you liked one of the answer please consider accepting it - Thank you! – vonjd Nov 13 '13 at 15:04
@vonjd: There are several answers that I like and upvoted. However, I think current answers still lack some details. For instance, even if estimating $\mu$ is the main driver behind real-world evolutions, if you just substitute $\mu$ by $r$, but all other parameters are left unchanged, the proposed evolution will not be calibrated to current market prices. Therefore, if calibration (i.e.: no arbitrage conditions) is still necessary to estimate real-world evolutions, which other steps are need once you have already estimated $\mu$? – sets Nov 13 '13 at 15:50

The risk-neutral measure $\mathbb{Q}$ is a mathematical construct which stems from the law of one price, also known as the principle of no riskless arbitrage and which you may already have heard of in the following terms: "there is no free lunch in financial markets".

This law is at the heart of securities' relative valuation, see this very nice paper by Emmanuel Derman and some part of this discussion.

In what follows, assume for the sake of simplicity

• existence of a risk-free asset ;
• deterministic and constant rates, with risk-free rate $r$ ;
• no dividends and no additional equity funding costs.

How to relate $\mathbb{Q}$ to $\mathbb{P}$: some useful concepts

The risk-neutral measure $\mathbb{Q}$ is a probability measure which is equivalent to $\mathbb{P}$ and under which the prices of (exchangeable) assets, discounted at the risk-free rate, turn out to be martingales.

1. If one assumes there is no free lunch in the real world (hence under $\mathbb{P}$), then the above definition (more specifically the "equivalent" part) suggests that there will be no free lunch under $\mathbb{Q}$ either. To convince yourself have a look at the accepted answer to this SE question. This answers your question concerning no arbitrage conditions.

2. The martingale property is convenient since it allows us to represent asset prices as expectations, which seems intuitive and natural. Indeed, it is well-known that if $X_t$ is a $\mathbb{Q}$-martingale then: $$X_0 = E^{\mathbb{Q}}[X_t \vert \mathcal{F}_0]$$

3. The adjective risk-neutral comes from the fact that, using a dynamic replication argument and under the assumption of no free lunch (+ market completeness, continuous trading, no frictions), one can show that the true performance of the stock simply disappears from the option valuation problem. Risk aversion thus disappears and only the risk-free rate $r$ remains. This is exactly what Black-Scholes-Merton showed and which earned them the Nobel prize in the first place, see below.

A simple example: the Black-Scholes model

Assume that the stock price $S_t$ follows a GBM under $\mathbb{P}$ $$\frac{dS_t}{S_t} = \mu dt + \sigma dW_t^{\mathbb{P}}\ \ \ (1)$$ where $\mu$ is the expected performance of the stock and $\sigma$ the annualised volatility of log-returns. This equation describes the dynamics of the stock in the real world.

Consider the pricing (we are still under in the real world) of a contingent claim $V_t = V(t,S_t)$ of which the only thing we know is that it pays out $\phi(S_T)$ to its holder when $t=T$ (generic European option). Now, consider the following self-financing portfolio:

$$\Pi_t = V_t - \alpha S_t$$

Using Itô's lemma along with the self-financing property yields: \begin{align} d\Pi_t &= dV_t - \alpha dS_t \\ &= \left( \frac{\partial V}{\partial t} + \frac{1}{2} \sigma^2 S_t^2 \frac{\partial^2 V}{\partial S^2}\right) dt + \left( \frac{\partial V}{\partial S} - \alpha \right) dS_t\\ \end{align}

The original argument of Black-Scholes-Merton is then that, if we can dynamically rebalance the portfolio $\Pi_t$ so that the number of shares held is continuously adjusted to be equal to $\alpha = \frac{\partial V}{\partial S}$, then the portfolio $\Pi_t$ would drift at a deterministic rate which, by absence of arbitrage opportunity, should match the risk-free rate.

Writing this as $d\Pi_t = \Pi_t r dt$ and remembering that we've picked $\alpha = \frac{\partial V}{\partial S}$ to reach this conclusion, we have

\begin{align} &d\Pi_t = \Pi_t r dt \\ \iff& \left( \frac{\partial V}{\partial t} + \frac{1}{2} \sigma^2 S_t^2 \frac{\partial^2 V}{\partial S^2} \sigma^2 \right) dt = \left( V_t - \frac{\partial V}{\partial S} S_t \right) r dt \\ \iff& \frac{\partial V}{\partial t}(t,S) + r S \frac{\partial V}{\partial S}(t,S) + \frac{1}{2} \sigma^2 S^2 \frac{\partial^2 V}{\partial S^2}(t,S) - rV(t,S) = 0 \end{align} which is the famous Black-Scholes pricing equation. Now, the Feynmann-Kac theorem tells us that the solution to the above PDE can be computed as: $$V_0 = E^\mathbb{Q}[ e^{-rT} \phi(S_T) \vert \mathcal{F}_0 ]$$ where under a certain measure $\mathbb{Q}$ $$\frac{dS_t}{S_t} = r dt + \sigma dW_t^{\mathbb{Q}}$$ which shows that $$\frac{V_t}{B_t} \text{ and } \frac{S_t}{B_t} \text{ are } \mathbb{Q}\text{-martingales}$$ with $B_t = e^{rt}$ representing the value of the risk-free asset we mentioned in the introduction. Notice how $\mu$ has completely disappeared from the pricing equation.

Because this Feynman-Kac formula very much resembles a magical trick, let us zoom in on the change of measure from a more mathematical perspective (the above was indeed the financial argument... at least for deriving the pricing equation, not for expressing its solution in martingale form).

Starting from $(1)$, let us define the quantity $\lambda$ as the excess-return over the risk-free rate of our stock, expressed in volatility units (ie its Sharpe ratio): $$\lambda = \frac{\mu - r}{\sigma}$$ Plugging this into $(1)$ gives: $$\frac{dS_t}{S_t} = r dt + \sigma (dW_t^{\mathbb{P}} + \lambda dt)$$ Now Girsanov theorem tells us that if we define the Radon-Nikodym of the change of measure as $$\left. \frac{d\mathbb{Q}}{d\mathbb{P}} \right\vert_{\mathcal{F}_t} = \mathcal{E}(-\lambda W_t^{\mathbb{P}})$$ then the process $$W_t^{\mathbb{Q}} := W_t^{\mathbb{P}} - \langle W^{\mathbb{P}}, -\lambda W^{\mathbb{P}} \rangle_t = W_t^{\mathbb{P}} + \lambda t$$ will emerge as a $\mathbb{Q}$-Brownian motion, hence we can write: $$\frac{dS_t}{S_t} = r dt + \sigma dW_t^{\mathbb{Q}}$$

Okay, this might seem even more magic to you than earlier, but there is a rigorous mathematical treatment behind don't worry.

Anyway, an interesting feature of writing and manipulating the Radon-Nikodym derivative is that one can eventually show that:

$$V_0 = E^{\mathbb{Q}} \left[ \left. \frac{V_T}{B_T} \right\vert \mathcal{F}_0 \right] = E^{\mathbb{P}} \left[ \left. \frac{V_T}{B_T} \mathcal{E}(-\lambda W_T^\mathbb{P}) \right\vert \mathcal{F}_0 \right]$$

where I have used the Bayes' rule for condition expectations, with $$X := V_T/B_T,\ \ \ f := \frac{d\mathbb{Q}}{d\mathbb{P}} \vert \mathcal {F}_T = \mathcal{E}(-\lambda W_T^{\mathbb{P}}),\ \ \ E^\mathbb{P}[f \vert \mathcal{F}_0 ] = 1$$

The above result is extremely interesting and can here be re-expressed as

$$V_0 = E^{\mathbb{Q}} \left[ e^{-rT} \phi(S_T) \vert \mathcal{F}_0 \right] = E^{\mathbb{P}} \left[ e^{-\left(r+\frac{\lambda^2}{2}+\frac{\lambda}{T} W_T^{\mathbb{P}}\right)T} \phi(S_T) \vert \mathcal{F}_0 \right]$$

This shows that, under BS assumptions:

• The option price can be calculated as an expectation under $\mathbb{Q}$ in which case we discount cash flows at the risk-free rate.
• The option price can also be calculated as an expectation under $\mathbb{P}$ but this time we need to discount cash flows based on our risk-aversion, which transpires through the market risk premium $\lambda$ (which depends on $\mu$).

Therefore, in addition to use expected returns, which other adjustment might be needed in order to estimate real-world probabilities?

You need to use a stochastic discount factor accounting for the risk aversion, see above and further remarks below.

Estimating real-world probabilities assuming BS

You have different possibilities here. The most straightforward is to calibrate your diffusion model to observed time series. When doing that, you'll get an estimate for $\mu$ and $\sigma$ in the GBM case. Now given what we just said earlier, you must be very careful when pricing under $\mathbb{P}$: you cannot discount at the risk-free rate.

It's more complicated than that when you choose another model than BS

The relationship: $$V_0 = E^{\mathbb{Q}} \left[ \frac{V_T}{B_T} \vert \mathcal{F}_0 \right] = E^{\mathbb{P}} \left[ \frac{V_T}{B_T} f \vert \mathcal{F}_0 \right]$$ with $$f = \left. \frac{d\mathbb{Q}}{d\mathbb{P}} \right\vert_{\mathcal{F}_T}$$ will hold (under mild technical conditions).

Compared to the risk-free discount factor $$DF (0,T):=1/B_T$$ the quantity $$SDF (0,T):=f/B_T$$ is best known as a Stochastic Discount Factor (maybe you've already heard about SDF models, this is precisely that) and we can write, without loss of generality:

$$V_0 = E^{\mathbb{Q}} \left[ DF (0,T) V_T \vert \mathcal{F}_0 \right] = E^{\mathbb{P}} \left[ SDF (0,T) V_T \vert \mathcal{F}_0 \right]$$

The problem is that, depending on the model assumptions you use, you cannot always have a simple and/or unique form for $f$ (hence $SDF (0,T)$) as it used to be the case in BS.

This is notably the case for incomplete models (i.e. models that include jumps and/or stochastic volatility etc.). So now you understand why when we need models to price options, we directly calibrate them under $\mathbb{Q}$ and not on time series observed under $\mathbb{P}$.

-
Indeed. This is the type of answer that I was looking for. Are you aware of any book or paper with numerical examples on SDFs to develop some intuition on its practical implementation? – sets yesterday
None off the top of my head, sorry. But you'll probably find plenty of good references via Google. It really depends on what your final objective is. Could you clarify that? – Quantuple yesterday
Also there seems plenty to related questions on SE, just use the "stochastic discount factor" keywords, see here for instance: quant.stackexchange.com/questions/15674/…. – Quantuple yesterday
Well just use the equation I mentioned in the BS framework, along with fixed values of $\mu$, $\lambda$, $\sigma$ and $T$. You can compute both expectations with a simple Monte Carlo simulation for instance. – Quantuple yesterday

You may want to consider splitting two important, yet very different concepts:

Pricing a derivative security with contingent payoff and forecasting an asset.

• Pricing a derivative can be achieved through setting up a hedge portfolio and track its evolution and "value" at any point in time before the derivative security pays off. Risk-neutral pricing is a handy tool to accomplish that. In most all cases do you need to possess knowledge of the underlying price dynamics which most likely depend on one or more random components, such as Brownian Motion.

• Estimating the probability of a non-contingent asset (such as a stock) reaching certain thresholds can be done entirely without the construct of any risk neutral probability measure. All you need is a pricing model and a parameter set (which you could estimate or derive from a fit to historical data) and run a simple Monte Carlo simulation. No need for risk neutral probabilities at all.

My point is that the concept of risk neutral pricing is not necessary if you want to estimate the probability of an asset with non contingent payoff to reach certain price levels. Your question was which models could be used to estimate the probability of reaching such thresholds: You can setup a pricing model, whose parameters you fit to past data, and throw it into a MC pricer. Check how many of the paths reach your thresholds and derive your probability. That is an example where you use real-world parameters to estimate a real-world probability.

EDIT (in response to edited question)

Calibration -> Calibrate real-world probabilities to historical data and models, incorporating risk-neutral probabilities, to current market prices.

No-arbitrage conditions: No they cannot be relaxed, and why would you want to do that? You look for a self-consistent model and if you calibrate to current market prices but throw overboard no-arbitrage conditions then you end up with incorrect probabilities because your model is distorted.

Historical fit: You can calibrate any model that incorporates real-world probabilities to historical data. Whether history repeats itself and whether your assumed risk premiums lead to the correct probabilities is an entirely different question.

Expected Returns: You do not have the choice when using real-probabilities; you cannot use the risk-free rate because in your world of real probabilities investors are risk averse and apply different utility curves, hence, you need to estimate risk premiums and expected returns instead of simply using a risk-free rate. As this is a pretty damning exercise it is the precise reason why risk-neutral probabilistic models are so attractive.

In short? A non risk-neutral probability model.

-
I am afraid that this - while of course true - doesn't really address the question of the OP: "If I want to estimate the "real world" probability of an asset reaching certain thresholds, which models and alternatives could be used?" So I guess it would be helpful to expand your comment "All you need is a pricing model and a parameter set (which you could estimate or derive from a fit to historical data) and run a simple Monte Carlo simulation." – vonjd Jun 20 '13 at 17:08
Fair, though I wanted to stress that risk neutral probabilities and estimating the probability of an asset with non-contingent payoff reaching certain thresholds are two very different and unrelated exercises. But will try to elaborate. Thanks – Matt Wolf Jun 20 '13 at 17:14
+1: Thank you :-) – vonjd Jun 20 '13 at 17:20
@MattWolf, I have just updated the question. Will be very helpful if you could elaborate your answer. – sets Jun 27 '13 at 11:43
@sets, I edited my answer to reflect your edited question – Matt Wolf Jun 28 '13 at 4:42

This is indeed one of the most difficult tasks to do (if not next to impossible).

I would say the standard reference is the following:
Expected Returns: An Investor's Guide to Harvesting Market Rewards by Antti Ilmanen

An abridged (but still about 170 pages long), yet more current - and free (!) version in different formats (pdf, mobi for the Kindle and epub) can be found here:
Expected Returns on Major Asset Classes by Antti Ilmanen

Addendum: A 8-page long summary of the main points can be found here:
Understanding Expected Returns by Antti Ilmanen

-
Why the downvote??? – vonjd yesterday

First of all, I must say that it's a very general question, and the answer can vary depending on type of assets you model.

In quant finance real world probabilities are generally used for risk management. It can be said, that in order to use real-world probabilities you have to calibrate your models to history. In order to obtain risk-neutral probabilities, you fit to market.

Simplest example - brownian motion for asset price. It is $\frac{dS}{S} = \mu dt + \sigma dW_t$ in real world and $\frac{dS}{S} = r dt + \sigma dW_t$ in risk-neutral world.

Where would you take $\mu$ from? The easiest way is to take history and estimate historical asset drift, or just calculate $\frac{1}{N}\sum_{i=1}^N\frac{S_{i+1}-S_i}{S_i}$.

Where would you take $r$ from? You just take current risk-free rate.

At the same time I must stress that quant finance models are IMHO unsuitable for long-term forecasting. In this case you have to seek for appropriate econometric model.

-