What is the Risk Neutral Measure?
I don't believe this has been answered on the internet well and with all the parts connecting.
So:
What is the risk neutral measure/pricing?
Why do we need it?
How we calculate the risk neutral measure or probabilities in practice?
What connection has risk neutral pricing to the drift of a SDE? Does this help with 3)?
Great answer given by Kevin. I would like to contribute an additional perspective. My experience with and my understanding of the Risk Neutral measure is entirely based on "no arbitrage" and "replication / hedging" arguments.
The way I would like to explain this view is via the following three-step construction:
(i) First, I want to build the intuition with a one-period discrete model: only a single stock and a risk-free account, no derivatives. The aim is to show that even without trying to price derivatives, one can create a mathematical object called a "risk-neutral probability measure", just by assuming no arbitrage in the model.
(ii) I then want to show that pricing a derivative by replication of its payoff with the underlying instrument and the risk-free rate instrument is equivalent to taking the expectation of the derivative pay-off under the risk-neutral measure & discounting it.
(iii) I then want to highlight that the discrete model converges to the well-known continuous Black-Scholes model.
I assume that today's stock price is $S_0$, and one period from now, the stock price can be $S_0 * u=S_u$ or $S_0 * d=S_d$, with $u$ and $d$ being "up" and "down" multiplicative factors. I assume that the risk-free rate is $r$.
Without imposing some conditions on $u$, $d$ and $r$, there might be some arbitrage opportunities. If for example $e^r>u$, I could short the stock at time $t_0$ and invest the proceeds $S_0$ into the risk-free account: in both future states at time $t_1$, I could then buy the stock back for less than my proceeds $S_0e^r$ from the risk free (because $S_u$ and $S_d$ would both be less than $S_0e^r$).
If, on the other hand, $e^r<d$, I could borrow exactly $S_0$ money at $t_0$ to buy the stock, and in both future states of the world at time $t_1$, the stock value would be higher than what I would have to repay ($S_0e^r$): so there would be arbitrage again.
Imposing $d \leq e^r \leq u$, will ensure no arbitrage in the one-period model.
Now I am going to perform the following algebraic manipulation:
$$ S_0 = \frac{S_0(u-d)}{(u-d)}= \\= \frac{1}{e^r}\frac{S_0(u-d)e^r}{(u-d)}= \\ =\frac{1}{e^r}\frac{S_0(u-d)e^r+(S_0ud - S_0ud)}{(u-d)}=\\= \frac{1}{e^r}\left( \frac{S_0ue^r -(S_0ud)}{u-d} + \frac{-S_0de^r+(S_0ud)}{u-d} \right)=\\=\frac{1}{e^r}\left(S_0u \left( \frac{e^r -d}{u-d} \right) + S_0d \left(\frac{u-e^r}{u-d} \right) \right)$$
The no arbitrage condition $d \leq e^r \leq u$ will result in the following bounds:
$$0 \leq \frac{e^r -d}{u-d} \leq 1$$
$$0 \leq \frac{u-e^r}{u-d} \leq 1$$
Furthermore:
$$ \frac{e^r -d}{u-d} + \frac{u-e^r}{u-d} = 1 $$
Let's call $\frac{e^r -d}{u-d}:=p_u$ and $\frac{u-e^r}{u-d}:=p_d$. In the one period model, the stock going up and the stock going down are two different states of the world, i.e. there is no "intersection" between these states in the probabilistic sense. Therefore $p_u$ and $p_d$ are additive over disjoint sets and they are within the zero-one range, therefore mathematically, these parameters qualify as a probability measure.
Rewriting the algebraic manipulation above in terms of $p_u$ & $p_d$ yields the following:
$$ S_0 = \frac{S_u p_u + S_d p_d}{e^r} = \frac{1}{e^r}\mathbb{E} [S_1] $$
Also notice that in the entire construction above, we did not talk about the probabilities of the stock going up or down. Every market participant might have his or her Bayesian view of the world with different probabilities assigned to the stock going up or down. But the risk-neutral measure is agreed upon by the market as a whole as a consequence of no arbitrage.
This also brings up an interesting point: in my view, the risk neutral probabilities are probabilities only in the "mathematical object" sense. They do not actually represent "likelihoods", in the sense that we human beings like to interpret probabilistic events with.
Let's assume we want to price a derivative on the stock with pay-off function $V(S_t)$ (could be a forward, option, whatever). The derivative pay-off in the two states will trivially be $V(S_u)$ and $V(S_d)$. We have two states, two underlying instruments: let's try to replicate the derivative pay-off in both states ($x$ is the number of stocks and $y$ is the amount invested in the risk-free account: I want to replicate the derivative pay-off in both states with $x$ stocks and $y$ risk-free investment):
$$ (i) x S_u + ye^r = V(S_u) $$ $$ (ii) x S_d + ye^r = V(S_d) $$
Solving gives:
$$ x = \frac{V(S_u)-V(S_d)}{S_0(u-d)} $$
$$ y = \frac{uV(S_d)-dV(S_u)}{(u-d)} \frac{1}{e^r} $$
Therefore the derivative price at time $t_0$ is the $x$ amount of the stock + $y$ amount invested in the risk-free account:
$$ V(S_0,t_0) = x*S_0 + y*1 = \\ = \frac{V(S_u)-V(S_d)}{S_0(u-d)}*S_0 + \frac{uV(S_d)-dV(S_u)}{(u-d)} \frac{1}{e^r}*1$$.
The above evaluates to:
$$\frac{1}{e^r}\left(V(S_u) \left( \frac{e^r -d}{u-d} \right) + V(S_d) \left(\frac{u-e^r}{u-d} \right) \right) $$
Notice that again we can write $\frac{e^r -d}{u-d}:=p_u$ and $\frac{u-e^r}{u-d}:=p_d$, where notably $p_u$ and $p_d$ are the same as in Part 1 above, Therefore, instead of having to compute the replication portfolio weights $x$ and $y$, the derivative can be priced as:
$$ V(S_0,t_0) = \frac{1}{e^r}\left(V(S_u) p_u + V(S_d) p_d \right) = \\ = \frac{1}{e^r} \mathbb{E}[V(S_1,t_1)]$$
Hopefully, by now you can see where I am going with this: the risk-neutral measure pricing technique has the following features:
(A) Is a consequence of no-arbitrage assumptions in the model
(B) Taking the expectation of a derivative pay-off and discounting it to today is the equivalent of: computing "replication portfolio" weights at each time-step, and pricing the derivative using these replicating weights at time $t_0$.
Extending the one-period model leads to a multi-period "binomial tree" discrete model. Pricing a derivative on a multi-period tree would require working "backwards" from the terminal pay-off and computing the replicating portfolio pay-off at each node. Alternatively, the more convenient way is to use the risk-neutral expectation of the terminal pay-off and discounting it to "today": as that will produce the same result (as shown above) and will save us having to worry about the replicating portfolio weights.
There are multiple papers online showing how the binomial tree model converges to the Black-Scholes formula when the number of steps tends to infinity as $\delta t$ tends to zero (for example a great paper by John Hull here or alternatively this paper here).
As a sketch, we can show that the multi-period Binomial model for the stock converges to the well-known continuous Geometric Brownian Motion (GBM) model (which in turn can be used to derive the Black-Scholes formula directly by applying the risk-neutral expectation to the option pay-off at maturity where the stock process is simulated with GBM).
For the multi-period Binomial model, we have:
$$S_n=S_0u^kd^{n-k}$$
with $k\sim Bin(n,p)$. The "Cox-Ross-Rubinstein" parameters for "up" and "down" are then set as follows: $u:=e^{\eta\frac{T}{n}+\sqrt{\frac{T}{n}}\sigma}$, $d:=e^{\eta\frac{T}{n}-\sqrt{\frac{T}{n}}\sigma}$.
It is well known that the Binomial distribution converges to Normal, in fact, using CLT, we get:
$$\lim_{n\to\infty}k\xrightarrow{d}N(np,\sqrt{np(1-p)})$$
Going back to the equation for $S_n$, taking a log and substituting for $u$ and $d$, we get:
$$ln \left( \frac{S_n}{S_0} \right) = k \left( 2 \sqrt{\frac{T}{n}} \sigma \right) + n\left(\eta \frac{T}{n}-\sqrt{\frac{T}{n}}\sigma\right)$$
Which gives:
$$\mathbb{E}\left[ln \left( \frac{S_n}{S_0} \right)\right]=\sqrt{Tn}\sigma(2p-1)+\eta T$$
$$Var\left(ln \left( \frac{X_n}{X_0} \right) \right)=4T\sigma^2p(1-p))$$
So we see that:
$$\lim_{n\to\infty}\left( \frac{X_n}{X_0} \right)\xrightarrow{d}N(\sqrt{Tn}\sigma(2p-1)+\eta T,2\sqrt{T}\sigma\sqrt{p(1-p))}$$
Taking $p=0.5$, we get:
$$\lim_{n\to\infty}ln\left(\frac{X_n}{X_0}\right)\xrightarrow{d} N\left(\eta T,\sigma \sqrt{T}\right)$$
If we now set $\eta:=\mu -0.5\sigma^2$, we see that the discrete multiperiod Binomial model for the stock price converges exactly to the continuous Geometric-Brownian-Motion model in distribution.
Another interesting fact to note is that the replicating weight of the stock in part 1, i.e. $x$, converges to $N(d_1)$, i.e. the instantaneous option Delta.
I will conclude by producing the same summary as Kevin, but with the following additional comments:
Summary
Risk-neutral probability measures are artificial measures (agreed) made up of risk-aversion (SDF) and real-world probabilities (disagree here: don't think risk-aversion comes into it. I see it as an artificial measure entirely created by assuming the existence of no-arbitrage and completeness).
Derivatives can be priced relative to underlying assets. This hedging price can be computed as expectation with respect to the risk-neutral probability measure (agreed). Equivalent martingale measures are deeply related to the absence of arbitrage and completeness (agreed: I would say they are not just deeply related to these, they are the consequence of these).
The risk-neutral density can be estimated from observed market data (agreed: i.e. twice differentiating the Implied Vol surface with respect to strike). The risk-neutral framework connects many different approaches to derivatives pricing
How would we price assets without the measure $\mathbb Q$? Well, we would start with some version of the Euler equation $P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]$, where $M$ is the stochastic discount factor (SDF). This equation holds under very weak assumptions (law of one price) and uses real-world probabilities. So, we take the return in every future state of nature multiply with the SDF to account for riskiness and weight this product by real world probabilities. This gives the current price of any asset (underlying and derivative).
The main problem: What is the SDF? In order to find the SDF, we need a general equilibrium model (such as C-CAPM or CAPM) which requires us to make assumptions about investor's utility function (simple CRRA or recursive Epstein-Zin?) etc. There are many different proposed SDFs in the asset pricing literature. And there is disagreement which models fits the data best.
Recalling the Euler equation $P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]$, the idea is to merge the SDF into the expectation (i.e. changing the probabilities associated to the expected value) which allows us to write $$P_t=\mathbb{E}_t[M_{t+1}P_{t+1}]=\frac{1}{1+r}\mathbb{E}_t^Q[P_{t+1}].$$ Here, I kind of assume discrete time steps and discrete compounding at a fixed rate $r$ but it of course equally applies in a time continuous setting.
The benefits are clear. We can now compute today's price of an asset by computing the expectation of the future price (or payoff) and discount this expectation at the risk-free rate. Importantly, we do not need to specify a utility function and an SDF anymore. We only need to find these new probabilities (the artificial measure $\mathbb Q$) and compute the expectation with respect to this new measure. In particular, the derivatives price will not depend on real-world probabilities.
What's the intuition about $\mathbb Q$? Well, it's a merger of real-world probabilities with the stochastic discount factor. This means $\mathbb Q$ adjusts the probabilities of outcomes by joining them with risk preferences (state prices). The new probabilities would then correspond to a world where all investors are risk-neutral (in which investors do not ask for risk premia and discount every cash flow at the risk-free rate $r$). This explains why $\mathbb Q$ is also called risk-neutral measure (i.e. $M$ represents risk and it vanished, what's left as discount factor is the risk-free rate of return, $r$). Risk-neutrality means being indifferent between any gamble and its expected payoff.
In line with the SDF interpretation, risk-neutral probabilities of bad events (declining stock prices) are higher than their corresponding real-world probabilities. On the other hand, risk-neutral probabilities deflate the likelihood of good events (increasing stock prices). As a result, the risk-neutral density is skewed to the left (negative skewness).
Another common term for the risk-neutral measure is equivalent martingale measure. Equivalent simply means that both measures agree on which events have zero probability. A martingale is a (integrable and adapted) stochastic process which models a fair game, i.e. $\mathbb{E}[X_t|\mathcal{F}_s]=X_s$ (the best prediction for the future value $X_t$ given the knowledge $\mathcal{F}_s$ at time $s$ is the value at time $s$ itself, $X_s$.
Stock prices have some real world drift $\mu$ which rewards investors for holding this risky assets. Typically, $\mu>r$. In a risk-neutral world, agents do not care about risk and do not ask for such a risk premium. Thus, in a market with risk-neutral investors, stocks (and every other assets) return the risk-free rate $r$. If we now discount stock prices using the risk-free asset as numéraire, we eliminate the drift and the discounted stock price becomes a $\mathbb Q$-martingale. Importantly, this only holds in the artificial risk-neutral world. In the real world, stocks do bear an equity risk premium and their price is not a martingales.
By the tower law, discounted derivative prices are martingales as well. They are defined by $V_t=e^{-r(T-t)}\mathbb{E}^\mathbb{Q}[X|\mathcal{F}_s]$, where the random variable $X$ is the future payoff (we actually assume the absence of arbitrage here for $\mathbb{Q}$ to exist, see below).
The above derivative price does not depend on the drift of the stock price. That's an important insight. In praxis this is great because we do not need to estimate the mean rate of return of a stock. Furthermore, even if two agents had completely different estimates for this drift, they could still agree on a fair price of an option written on this stock because the drift does not matter. In some extreme case, if $K=0$, a call option would just pay the stock and that could be replicated by just buying the stock. No need to consider the drift. Alternatively, at the money (forward), put and call options have the same value. If the drift mattered, either put or call options would increase in value, violating this relationship.
The fundamental theorems of asset pricing are key results in finance linking the market properties arbitrage and completeness to risk-neutral probability measures.
In simple terms, an arbitrage (free lunch) is a self-financing trading strategy which does not require any funds at inception and yields a non-negative payoff in the future with a positive probability of an actual gain (strictly positive payoff). A complete market is a market in which any reasonable payoff can be replicated.
Because the set of all equivalent martingale measures is convex, either no EMM exists (there exist arbitrage strategies), or precisely one EMM exists (market is free of arbitrage and complete) or infinitely many EMMs exist (market is free of arbitrage but incomplete). It is impossible to only have 2, 42 or 3141592 EMMs.
The assumption that no arbitrage exists is an easy one to accept. We thus know that at least one equivalent martingale measure exists. The question is completeness. This is debatable (Is volatility risk tradable? What about jump risk? etc). If infinitely many EMMs exist, then option prices are not unique, i.e. there exist pricing intervals whose elements all represent valid, arbitrage-free derivative prices. In these cases, one either neglects some risk sources (pretending the market is complete) by choosing a corresponding model or one implements some minimum variance hedging strategy, accepting that there's not one true price.
Unfortunately, absence of arbitrage and completeness are a bit at odds. The more asset payoffs there are, the more complete the market is but arbitrage possibilities are also likelier to exist.
A key element in no-arbitrage pricing theory are linear pricing functionals which extend hedging prices and are monotone (higher payoffs lead to higher prices). By Riesz' Representation Theorem, such a linear functional can be represented by an inner product, i.e. there exists some random variable $M$ such that the pricing functional $\pi$ can be written as $\pi(X)=\mathbb{E}^\mathbb{P}[MX]$ for any payoff (contract) $X$. This $M$ is of course the SDF.
So, if no arbitrage strategy exists, we have (at least) one linear pricing functional. This functional gives rise to an SDF and an SDF can be merged with real-world probabilities to give a risk-neutral measure. This chain directly links hedging prices to EMMs. In fact, there’s an one-to-one relationship between pricing functionals and EMMs. Thus, just like EMMs, the set of pricing functionals is convex. If markets are complete, then the SDF and EMM are unique. As a result, they have to coincide with the simple hedging price.
Agreeing with hedging prices further motivates why individual risk preferences do not enter the risk-neutral pricing framework. You price assets relative to each other. You assume you know the prices of some basic (primitive) assets (from some equilibrium model) and then you price further assets (derivatives) by trading the basic options. This is possible by the absence of arbitrage. The prices of the original assets already incorporate risk-aversion etc. So, by relative pricing (hedging), you do not need to include preferences again.
Breeden and Litzenberger (1978) show that $$\mathbb{Q}[\{S_T\geq \kappa\}] = -e^{rT}\frac{\partial C(S_0,K,T)}{\partial K}\bigg|_{K=\kappa},$$ which in turn means that the risk-neutral density can be extracted as follows $$q_T(\kappa) = e^{rT}\frac{\partial^2 C(S_0,K,T)}{\partial K^2}\bigg|_{K=\kappa}.$$
We can thus use observed option prices, $C(S_0,K,T)$, to estimate the risk-neutral density $q_T$. Clearly, the risk-neutral density changes over time and with option maturity.
The biggest problem with this approach is that option prices are needed at every positive strike. Around ATM strikes, there are enough liquid options which can easily be interpolated but at extreme strikes (very OTM and very ITM), data becomes an issue. Using the put-call parity, one typically focusses on more liquid OTM options but estimating the tails of the risk-neutral density is difficult and one often uses semi-parametric approaches which assume some functional form for the tails.
The risk-neutral pricing equation $P_t=\frac{1}{1+r}\mathbb{E}^\mathbb{Q}_t[P_{t+1}]$ is key to understand (almost) all numerical methods used in finance:
I just saw that @JanStuller's answer disagrees with me relating risk aversion to the risk-neutral measure. He is, of course, absolutely right. We don't need to do this. If the law of one price holds, then a stochastic discount factor (and state prices) exists and prices are linear functionals. If the market is additionally free of arbitrage, then the SDF (and every state price) is strictly positive (in each state of nature, not just almost surely). In this case, we can also define a risk-neutral measure by merging SDF and real-world probabilities to obtain an equivalent martingale measure. The state price approach leads to the pricing by hedging/replication in Jan's answer.
So far the theory. Nothing of this depends on equilibrium, risk aversion or utility maximisation. In this sense, Jan is absolutely right and we can link SDF, risk-neutral measure and state prices without further assumptions. However, isn't the question how do we explain economically what's going on? What does the SDF measure? How do risk-neutral probabilities differ from real-world probabilities? What's the intuition behind this artificial measure? Why is the risk-neutral measure called risk-neutral if it has nothing to do with risk aversion? To answer these questions, we need to think deeper about the SDF. It turns out that $P=\mathbb{E}^\mathbb{P}[MX]$ is just the first-order condition of an investor who maximizes his utility. The SDF $M$ is then indeed a function of his risk-aversion. Ultimately, the SDF prices every asset -- and that price should reasonably be related to preferences, tastes and attitudes towards risk.
So the bottom line is: the existence of a risk-neutral measure is utterly independent of utility functions etc. However, in order to make sense of the measure change in economic terms and to build up intuition, I think it's wise to include some financial economics.
I believe the other answers are nearly exhaustive; but here's a bit of intuition I'd like to add:
Think of the decision (= equilibrium price) of a market as:
Decision = f(probabilities, risk aversion)
where probabilities are the chances of various events happening, and risk aversion is the taste preference of the market.
Now it turns out that the 'iso-curve' always has a point where risk aversion is 0. That is, for some probabilities p and risk aversion a, I can find probabilities p' such that:
Decision(p,a)=Decision(p',0).
That is, I can always look at the market as if it was risk neutral (no risk aversion), provided I play around with the possibilities in a clever way (p changes to p').
For intuition into this, let us now try to have a risk neutral and a risk averse person arrive at the same 'decision' of investing:
The decision of a risk averse person at market probabilities of a up and down move (0.5,0.5) can be replicated by a risk neutral person at fictitious probabilities (0.4,0.6). [Ignore the specific values, just note that I've increased the probability of the down move].
This is because the averse person is much more 'sensitive' to the down move gives it a 'higher weight' in his decision making process. So, I need to incentivize him with a lower probability of a down move. However, the risk neutral person is much less sensitive and will make the same 'decision' at higher probability of a down move. This tells me that risk aversion and probability are 2 sides of the same coin, and can be inter-played with without affecting the 'decision (prices)'.
So, the risk averse man can be thought of as a risk neutral man by just accommodating the risk aversion in the probabilities.
This is what the risk neutral measure achieves: a change in probabilities of events, without compromising the market decision.
The above can always be done if there is no arbitrage in the market.
The derivative pricing thing comes into the picture if you impose completeness - that everything is replicable. That pins down p' - these probabilities are now 'unique'. This is why we use them in derivative pricing, because now pricing payoffs is trivial - as we only need to value a payoff using expectation (as a risk neutral person does).
I think other answers have provided a very exhaustive explanation on what is the risk-neutral measure, why do we need it with heavy formulas. I'd like to provide an intuitive view for people who like some real and simple numbers:
Not necessary, when you price a derivative, you don't have to adapt it, but it will end up to the same result as if you consider risk-neutral measure. Let me given an example of using the real probability:
Let's use a one-step binomial model as an example. Suppose there is a stock $S$ worth \$10 today, and it has a probability of $p$ to reach \$11 tomorrow, or probability of $1-p$ to reach $9. For simplicity, we assume the risk-free interest rate to be 0.
Now, you have a call option $C$ with strike price to be \$9, so you have a chance of $p$ to gain \$2 and $1-p$ to gain nothing. Now, the question is how do you like to price the option?
An intuitive way is to calculate $2 p + 0 (1 - p)$ as the price, so the price goes up with $p$ increases. However, is this the optimized price? Can we use a lower price to make a portfolio behaving the same as the option $C$? One can verify that if you borrow \$9 from the bank (assume no interest) and buy one share of the stock $S$, your return tomorrow is the same as if you buy an option $C$. How much do you spend in this portfolio? \$1, and this cost is independent of the value of $p$.
See, we didn't introduce any risk-neutral assumption here, and $p$ can be any value. Since the method does not need to know $p$, why not just assume the expectation of the stock to be the same as the risk-free rate if this assumption can reduce your computation cost?
The math (see other answers for the detailed proof) shows that when $p$ happens to make the expected return of a stock the same as the risk-free rate, the option price can be computed easily as $p C_u + (1 - p) C_d$, where $C_u$ and $C_d$ are the return of the option when the stock goes up and down, respectively, ignoring the discount rate in the binomial model. Using the above example, $p$ will be 0.5, and the option price is simply $(2 + 0) * 0.5 = 1$.
This is more like a method than some assumption on the real world. Note that, \$1 is only the upper bound of the price of the call option $C$. We prove there is one way to achieve this price, but this is not necessary the optimized (cheapest) way.
No arbitrage.
Still use the above example. Instead, Stock $S$ this time has a probability of $p$ to reach \$13 and $1-p$ to \$11. Similarly, when considering the risk-neutral measure, we first calculate $p$ making the expected return of $S$ to be the risk-free rate (which is 0 in this case), and $p = -\frac{1}{2}$. We may still use $p C_u + (1 - p) C_d$ to price an option. The corresponding option price is $4 \times (-\frac{1}{2}) + 2 \times \frac{3}{2} = 1$. That is, you can still borrow \$9 from the bank and add your \$ 1 to buy $S$, which will give you the same outcome as option $C$, so the option price from the risk-neutral measure is still \$1.
However, a negative probability is meaningless in general. If the return rate distribution of a stock is a continual distribution, like a Gaussian. Then, it is unlikely to make the expected return rate the same as the risk-free rate, so the risk-neutral measure will not work. In other words, "no arbitrage" requires at least one possible return to be lower than the risk-free rate, so that you can find a return rate distribution to satisfy the condition of the risk-neutral measure.
