It is not simply supply and demand. There was supply and demand for options before October 1987, yet, options traded largely without a skew. Supply and demand plays a role, but its a secondary consideration.
Implied vol is a way of turning an option price into a comparable number (its annualized for example). The theory to construct vol is based on the world of Black Scholes (its assumptions). That holds for all sorts of markets. If you construct vol surfaces from equity markets (predominantly American and price quoted), one usually de-Americanises the option before building a vol surface. Many OTC markets (e.g. rates, FX) mostly quote directly in IVOL.
Black Scholes implies normally distributed stock returns, whereas real (stock) returns are negatively skewed and have fatter tails because:
stocks (or other underlyings) tend to move down faster than they move up, so the left side has a fatter tail than the right side - known as skewness
extreme price movements in both directions ( called outliers) are more common than the normal distribution suggests, so both tails are fatter than a normal distribution would suggest - known as kurtosis
Traders use different vols to price options (on stocks) where returns tend to move differently than the normal distribution suggests, to more accurately represent the stocks movement.
FX is the "cleanest" way volatilities are quoted in my opinion. The quotes come as ATM DNS (delta neutral straddle), RR (Risk Reversals) and BF (Butterflies). The screenshots below will ignore quite a few technicalities; ATMD will not be 50D as the stylized example suggests, and is be explained here. IVOL itself affects the value of delta and many fx pairs are quoted as delta premium included. BF itself can be quoted differently.
In a nutshell,
- ATM determines the level (you can think of it as the Black Scholes IVOL for a specific tenor),
- RR the skew (how its tilted, towards OTM pits in the example below) and
- BF the kurtosis (how pronounced the general wings are).
Using a somewhat simplified method from Malz, one can quickly demonstrate this with a few lines of code in Julia.
Black Scholes looks like this (RR and BF set to 0).
A 25D RR is quoted as 25D Call - 25D Put vol; $RR_{25D}=C-P $. If OTM puts are more expensive (higher vol), it means that the RR is negative and the VOL surface is higher for OTM puts (to the left).
A 25D BF is quoted as $BF_{25D}=((C+P))/2-ATM $ in vol quotation. If the average of Call and Put vols is higher than ATM (smaller is very rare), you account for kurtosis (fatter tails on both sides). I set RR to zero for convenience here.
Putting it all together creates the VOL surface.
Hence, the vol surface exists mainly because there are fat tails, skewness, heteroscedasticity, jumps (crashes), and so forth. None of these real world phenomena are featured in the Black Scholes formula. The market just developed ways to account for many of the shortcomings of Black Scholes.
All these factors (as well as supply and demand) increase the market price of OTM puts (and calls, depending a bit on markets though) and therefore translates into higher implied volatilities.