Life Without a Risk-Neutral Measure
How would we price assets without the measure $\mathbb Q$? Well, we would start with some version of the Euler equation $P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]$, where $M$ is the stochastic discount factor (SDF). This equation holds under very weak assumptions (law of one price) and uses real-world probabilities. So, we take the return in every future state of nature multiply with the SDF to account for riskiness and weight this product by real world probabilities. This gives the current price of any asset (underlying and derivative).
The main problem: What is the SDF? In order to find the SDF, we need a general equilibrium model (such as C-CAPM or CAPM) which requires us to make assumptions about investor's utility function (simple CRRA or recursive Epstein-Zin?) etc. There are many different proposed SDFs in the asset pricing literature. And there is disagreement which models fits the data best.
Intuition
Recalling the Euler equation $P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]$, the idea is to merge the SDF into the expectation (i.e. changing the probabilities associated to the expected value) which allows us to write $$P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]=\frac{1}{1+r}\mathbb{E}_t^Q[P_{t+1}].$$ Here, I kind of assume discrete time steps and discrete compounding at a fixed rate $r$ but it of course equally applies in a time continuous setting.
The benefits are clear. We can now compute today's price of an asset by computing the expectation of the future price (or payoff) and discount this expectation at the risk-free rate. Importantly, we do not need to specify a utility function and an SDF anymore. We only need to find these new probabilities (the artificial measure $\mathbb Q$) and compute the expectation with respect to this new measure. In particular, the derivatives price will not depend on real-world probabilities.
What's the intuition about $\mathbb Q$? Well, it's a merger of real-world probabilities with the stochastic discount factor. This means $\mathbb Q$ adjusts the probabilities of outcomes by joining them with risk preferences (state prices). The new probabilities would then correspond to a world where all investors are risk-neutral (in which investors do not ask for risk premia and discount every cash flow at the risk-free rate $r$). This explains why $\mathbb Q$ is also called risk-neutral measure (i.e. $M$ represents risk and it vanished, what's left as discount factor is the risk-free rate of return, $r$). Risk-neutrality means being indifferent between any gamble and its expected payoff.
In line with the SDF interpretation, risk-neutral probabilities of bad events (declining stock prices) are higher than their corresponding real-world probabilities. On the other hand, risk-neutral probabilities deflate the likelihood of good events (increasing stock prices). As a result, the risk-neutral density is skewed to the left (negative skewness).
Martingales
Another common term for the risk-neutral measure is equivalent martingale measure. Equivalent simply means that both measures agree on which events have zero probability. A martingale is a (integrable and adapted) stochastic process which models a fair game, i.e. $\mathbb{E}[X_t|\mathcal{F}_s]=X_s$ (the best prediction for the future value $X_t$ given the knowledge $\mathcal{F}_s$ at time $s$ is the value at time $s$ itself, $X_s$.
Stock prices have some real world drift $\mu$ which rewards investors for holding this risky assets. Typically, $\mu>r$. In a risk-neutral world, agents do not care about risk and do not ask for such a risk premium. Thus, in a market with risk-neutral investors, stocks (and every other assets) return the risk-free rate $r$. If we now discount stock prices using the risk-free asset as numéraire, we eliminate the drift and the discounted stock price becomes a $\mathbb Q$-martingale. Importantly, this only holds in the artificial risk-neutral world. In the real world, stocks do bear an equity risk premium and their price is not a martingales.
By the tower law, discounted derivative prices are martingales as well. They are defined by $V_t=e^{-r(T-t)}\mathbb{E}^\mathbb{Q}[X|\mathcal{F}_s]$, where the random variable $X$ is the future payoff (we actually assume the absence of arbitrage here for $\mathbb{Q}$ to exist, see below).
The above derivative price does not depend on the drift of the stock price. That's an important insight. In praxis this is great because we do not need to estimate the mean rate of return of a stock. Furthermore, even if two agents had completely different estimates for this drift, they could still agree on a fair price of an option written on this stock because the drift does not matter. In some extreme case, if $K=0$, a call option would just pay the stock and that could be replicated by just buying the stock. No need to consider the drift. Alternatively, at the money (forward), put and call options have the same value. If the drift mattered, either put or call options would increase in value, violating this relationship.
Fundamental Theorems of Asset Pricing
The fundamental theorems of asset pricing are key results in finance linking the market properties arbitrage and completeness to risk-neutral probability measures.
- The first theorem of asset pricing states that the absence of arbitrage is equivalent to the existence of at least one equivalent martingale measure.
- The second theorem of asset pricing states that a market is complete if and only if at most one equivalent martingale measure exists.
In simple terms, an arbitrage (free lunch) is a self-financing trading strategy which does not require any funds at inception and yields a non-negative payoff in the future with a positive probability of an actual gain (strictly positive payoff). A complete market is a market in which any reasonable payoff can be replicated.
Because the set of all equivalent martingale measures is convex, either no EMM exists (there exist arbitrage strategies), or precisely one EMM exists (market is free of arbitrage and complete) or infinitely many EMMs exist (market is free of arbitrage but incomplete). It is impossible to only have 2, 42 or 3141592 EMMs.
The assumption that no arbitrage exists is an easy one to accept. We thus know that at least one equivalent martingale measure exists. The question is completeness. This is debatable (Is volatility risk tradable? What about jump risk? etc). If infinitely many EMMs exist, then option prices are not unique, i.e. there exist pricing intervals whose elements all represent valid, arbitrage-free derivative prices. In these cases, one either neglects some risk sources (pretending the market is complete) by choosing a corresponding model or one implements some minimum variance hedging strategy, accepting that there's not one true price.
Unfortunately, absence of arbitrage and completeness are a bit at odds. The more asset payoffs there are, the more complete the market is but arbitrage possibilities are also likelier to exist.
Relationship to Hedging Prices
A key element in no-arbitrage pricing theory are linear pricing functionals which extend hedging prices and are monotone (higher payoffs lead to higher prices). By Riesz' Representation Theorem, such a linear functional can be represented by an inner product, i.e. there exists some random variable $M$ such that the pricing functional $\pi$ can be written as $\pi(X)=\mathbb{E}^\mathbb{P}[MX]$ for any payoff (contract) $X$. This $M$ is of course the SDF.
So, if no arbitrage strategy exists, we have (at least) one linear pricing functional. This functional gives rise to an SDF and an SDF can be merged with real-world probabilities to give a risk-neutral measure. This chain directly links hedging prices to EMMs. In fact, there’s an one-to-one relationship between pricing functionals and EMMs. Thus, just like EMMs, the set of pricing functionals is convex. If markets are complete, then the SDF and EMM are unique. As a result, they have to coincide with the simple hedging price.
Agreeing with hedging prices further motivates why individual risk preferences do not enter the risk-neutral pricing framework. You price assets relative to each other. You assume you know the prices of some basic (primitive) assets (from some equilibrium model) and then you price further assets (derivatives) by trading the basic options. This is possible by the absence of arbitrage. The prices of the original assets already incorporate risk-aversion etc. So, by relative pricing (hedging), you do not need to include preferences again.
Estimating the Risk-Neutral Density
Breeden and Litzenberger (1978) show that
$$\mathbb{Q}[\{S_T\geq \kappa\}] = -e^{rT}\frac{\partial C(S_0,K,T)}{\partial K}\bigg|_{K=\kappa},$$ which in turn means that the risk-neutral density can be extracted as follows $$q_T(\kappa) = e^{rT}\frac{\partial^2 C(S_0,K,T)}{\partial K^2}\bigg|_{K=\kappa}.$$
We can thus use observed option prices, $C(S_0,K,T)$, to estimate the risk-neutral density $q_T$. Clearly, the risk-neutral density changes over time and with option maturity.
The biggest problem with this approach is that option prices are needed at every positive strike. Around ATM strikes, there are enough liquid options which can easily be interpolated but at extreme strikes (very OTM and very ITM), data becomes an issue. Using the put-call parity, one typically focusses on more liquid OTM options but estimating the tails of the risk-neutral density is difficult and one often uses semi-parametric approaches which assume some functional form for the tails.
Relationship to Numerical Methods
The risk-neutral pricing equation $P_t=\frac{1}{1+r}\mathbb{E}^\mathbb{Q}_t[P_{t+1}]$ is key to understand (almost) all numerical methods used in finance:
- Finite differences: They are used to solve the PDEs which define derivative prices. By the Feynman-Kac theorem, the diffusion equations appearing in finance can be written as a conditional expectation - the risk-neutral price.
- Binomial trees: You directly approximate the (risk-neutral) evolution of the underlying asset and compute expectations backwards through time. In particular, real-world probabilities do not enter this formula, just like in the risk-neutral framework.
- Monte Carlo simulations: You simply simulate the future price (in a risk-neutral world), compute the average (expectation) of this price and discount back at the risk-free rate (you directly approximate the aforementioned risk-neutral pricing equation).
- Fourier methods: You again start with the risk-neutral pricing equation and just change the integration domain: instead of integrating with respect to the risk-neutral density, you simply integrate in the Fourier domain using characteristic functions. This is fully equivalent and often easier.
- Quadrature: You directly use the (risk-neutral) transition density of the underlying asset price and numerical integration to approximate the expectation (an integral) of the future price.
So, the risk-neutral pricing framework is in the centre of quantitative finance and connects (almost) all methods used in quantitative finance.
Do we need to consider risk aversion for a risk-neutral measure?
I just saw that @JanStuller's answer disagrees with me relating risk aversion to the risk-neutral measure. He is, of course, absolutely right. We don't need to do this. If the law of one price holds, then a stochastic discount factor (and state prices) exists and prices are linear functionals. If the market is additionally free of arbitrage, then the SDF (and every state price) is strictly positive (in each state of nature, not just almost surely). In this case, we can also define a risk-neutral measure by merging SDF and real-world probabilities to obtain an equivalent martingale measure. The state price approach leads to the pricing by hedging/replication in Jan's answer.
So far the theory. Nothing of this depends on equilibrium, risk aversion or utility maximisation. In this sense, Jan is absolutely right and we can link SDF, risk-neutral measure and state prices without further assumptions. However, isn't the question how do we explain economically what's going on? What does the SDF measure? How do risk-neutral probabilities differ from real-world probabilities? What's the intuition behind this artificial measure? Why is the risk-neutral measure called risk-neutral if it has nothing to do with risk aversion? To answer these questions, we need to think deeper about the SDF. It turns out that $P=\mathbb{E}^\mathbb{P}[MX]$ is just the first-order condition of an investor who maximizes his utility. The SDF $M$ is then indeed a function of his risk-aversion. Ultimately, the SDF prices every asset -- and that price should reasonably be related to preferences, tastes and attitudes towards risk.
So the bottom line is: the existence of a risk-neutral measure is utterly independent of utility functions etc. However, in order to make sense of the measure change in economic terms and to build up intuition, I think it's wise to include some financial economics.
Summary
- Risk-neutral probability measures are artificial measures made up of SDF (risk-aversion) and real-world probabilities.
- The SDF can be linked to state prices which pave the way for pricing by replication. The SDF can be linked to the risk-neutral measure via a change of measure. What does the SDF measure? That's an economic question. In any equilibrium model, the SDF reflects agent's attitude towards risk.
- Derivatives can be priced relative to underlying assets. This hedging price can be computed as expectation with respect to the risk-neutral probability measure.
- Equivalent martingale measures are deeply related to the absence of arbitrage and completeness
- The risk-neutral density can be estimated from observed market data
- The risk-neutral framework connects many different approaches to derivatives pricing