I read that an option prices is the expected value of the payout under the risk neutral probability. Intuitively why is the expectation taken with respect to risk neutral as opposed to the actual probabilty.
Intuitively why would risk neutral probability differ from actual probability?
The following is a standard exercise that will help you answer your own question.
Consider a one-period binomial lattice for a stock with a constant risk-free rate.
Determine the initial cost of a portfolio that perfectly hedges a contingent claim with payoff $uX$ in the upstate and $dX$ in the downstate (you can do this so long as the up and down price are different in your lattice).
Assuming there exists no portfolio that yields a profit without downside risk (assume no arbitrage) and that your economy is frictionless and competitive, show that any other price for the contingent claim, other than the initial cost of the replicating portfolio you found, would lead to the existence of a portfolio that yields a profit without downside risk. Pause and reflect on the fact that you have determined the price of any contingent claim without any mention of probability. However, don't forget what you assumed! What did you actually need to do what you just did?
Now that you know that the price of the initial portfolio is the "arbitrage free" price of the contingent claim, find the number $q$ such that you can express that price of the contingent claim as the discounted payoff in the up state times a number $q$ plus the discounted payoff in the downstate times the number $1-q$. Solve for the number $q$. Interpret the number $q$ as a probability and compute the expected value of the discounted stock with this probability. This should be the same as the initial price of the stock. Pause and reflect on the fact that you have determined the unique number $q$ between $0$ and $1$ such that the expected value (using $q$) of the discounted stock is the initial price and that you can compute the price of any contingent claim by computing its expected (using $q$) discounted payoff.
It is clear from what you have just done that if you chose any other number $p$ between $0$ and $1$ other than the $q$ and computed the expected (using $p$) discount payoff, then you would not recover the arbitrage free price (remember you have shown that any other price than the one you found leads to an arbitrage portfolio). This means that if you had a real world probability $p$ for your initial lattice, it is not the correct probability to use when computing the price.
You're missing the point of the risk-neutral framework.
The idea is as follows: assume the real probability measure called $\mathbb{P}$. The thing is, because investors are not risk-neutral, you cannot write that $v_0 = E_\mathbb{P} [ e^{-rT} V_T]$.
Using the Fundamental Theorem of Asset Pricing, you know that if the market is arbitrage-free, then there exists a probability measure $\mathbb{Q}$ such that $v_0 = E_\mathbb{Q} [ e^{-rT} V_T]$.
So what you do is that you define the probability measure $\mathbb{Q}$ sur that $v_0 = E_\mathbb{Q} [ e^{-rT} V_T]$ holds.
Obviously, they are not the same thing.
Risk neutral probability differs from the actual probability by removing any trend component from the security apart from one given to it by the risk free rate of growth. If you think that the price of the security is to go up, you have a probability different from risk neutral probability.
In very layman terms, the expectation is taken with respect to the risk neutral probability because it is expected that any trend component should have been discounted for by the traders and hence at any moment, there is no non-speculative reason to assume that the security is biased towards the upside or the downside.
An answer has already been accepted, but I'd like to share what I believe is a more intuitive explanation.
There are many risk neutral probabilities ... probability of a stock going up over period $T-t$, probability of default over $T-t$ etc. The intuition is the same behind all of them.
In reality, you want to be compensated for taking on risk. This is why corporate bonds are cheaper than government bonds. In risk neutral valuation we pretend that investors are stupid and are willing to take on extra risk for no added compensation. The reason is it make the math easier. The intuition is to follow.
Let's consider the probability of a bond defaulting:
Imagine a corporate bond with a real world probability of default of 1%. This 1% is based on the historical probabilities of default for similar grade bonds and obtained form a rating agency. If the bond defaults we get 40% of the par value.
If we try to price the bond using only the real world probability of default given above to calculate the expected value of this bond and then present value it, we will come up with the wrong price. In fact, the price will bee too high.
Why? Because the bond's price takes into consideration the risk the investor faces and various other factors such as liquidity. We've ignored these and only have part of the picture.
Enter risk-neutral pricing. Instead of trying to figure out these pieces we've ignored, we are simply going to solve for a probability of default that sets PV(expected value) to the current market price. This is called a risk neutral probability. Well, the real world probability of default was 1% and just using that to value the bond overshot the actual price, so clearly our risk-neutral probability needs to be higher than the real world one.
I've borrowed my example from this book. I think the author gives the best explanation I've seen https://books.google.ca/books?id=6ITOBQAAQBAJ&pg=PA229&lpg=PA229&dq=risk+neutral+credit+spread+vs+actuarial&source=bl&ots=j9o76dQD5e&sig=oN7uV33AsQ3Nf3JahmsFoj6kSe0&hl=en&sa=X&ved=0CCMQ6AEwAWoVChMIqKb7zpqEyAIVxHA-Ch2Geg-B#v=onepage&q=risk%20neutral%20credit%20spread%20vs%20actuarial&f=true.
Valuing an option in a risk-neutral world is essentially saying that the risk preferences of investors do not impact option prices. That seems strange at first: given that options are risky investments, shouldn't they be affected by investor's risk preferences?
The answer is no, and the reason is clear: we are valuing the option in terms of the underlying share, and not in absolute terms. The risk-preferences of investors get incorporated in the share price itself (for instance, a higher risk aversion would reduce the share price), and so we don't have to account for them again while valuing the option in terms of the underlying share.
We can reinforce the above point by putting it in slightly different words: Imagine breaking down our model into two levels -
1) A "formula" linking risk preferences to the share price.
2) A "formula" linking the share price to the option price.
When risk preferences change, corresponding changes only occur at the first level; the formula linking the share price to option price remains unaffected.
I think the classic explanation (any other measure costs money) may not be the most intuitive explanation but it is also the most clear in some sense and therefore does not really require a intuitive explanation.
That is to say: you could use any measure you want, measures that make sense, measures that don't but if the measure you choose is a measure different from the risk neutral one you will use money. Therefore, don't.
