- A linear relationship between expected returns and covariance with a risk factor is a necessary consequence of a linear asset pricing function
- In theory, a CAPM relationship can be derived when a pricing kernel $S$ is affine in the return of the market portfolio. Numerous, differentDifferent sets of assumptions lead to this affine relationship. Be aware also is that the CAPM is an empirical failure. Do NOTfailure; do not use the CAPM for empirical asset pricing.
ConsequenceA consequence of a linear asset pricing function
Let $p(X)$ be an asset pricing function that gives thetoday's price of a random payoff $X$. A straightforward assumption is that $p$ should be a linear functional. Then by the Riesz representation theorem, there exists a pricing kernel (aka stochastic discount factor) $S$ such that $p$ can be written as an inner product of $S$ and $X$.
$$ p(X) = \operatorname{E}[S X]$$
The basic intuition is that with linearity, you can go back and forth between (1) security prices and (2) state prices (prices for outcomes from the sample space $\omega \in \Omega$). With complete markets, state price density $S$ is uniquely determined; with incomplete markets, $S$ is not uniquely determined. An academic dream has been to use macroeconomic theory to derive a state price density that is empirically consistent with security prices.
Rearranging above to write as a classic, regression beta model
If $X$ is a return, then $p(X) = 1$. Some algebraic manipulation of the above equation (eg. see John Cochrane's book Asset Pricing (revised)) allows you to represent this as a regression beta model:
\begin{align*} \operatorname{E}[R_i] - R^f &= \frac{\operatorname{Cov}(R_i, S)}{\operatorname{Var}(S)} \lambda_S \\ &= \beta_{R_i, S} \lambda_S\end{align*}
A linear asset pricing function $p$ implies that expected excess returns $\operatorname{E}[R_i] - R^f$ are linear with the covariance of $R_i$ and stochastic discount factor $S$.
The whole academic asset pricing game is about figuring out what $S$ is.
(Note that some papers have taken potshots at linearity, eg. Lamont and Thaler (2003), "Can the Market Add and Subtract?").
If you're a math guy, something to have in the back of your head is that all this mean-variance frontier stuff is just classic linear algebra in disguise. It's not super intuitive, but it's not that complicated either. For a technical exposition, perhaps see Hansen and Richard (1987) or for more intuition, Cochrane's exposition in chapter 5.3.
An economic interpretation of $S$
Simple economic arguments say that pricing kernel $S$ should be given by the ratio of marginal utility of consumption:
$$ S_{t, t+1} = \frac{u'(C_{t+1})}{u'(C_t)}$$.
If $S$ is affine in the market return, then you get the CAPM relationship between expected returns and covariance with market returns. A lot of different sets of assumptions can get you there.
References
Cochrane, John, 2005 Asset Pricing (revised)