It is a not so well known fact that the Kelly criterion is only optimal in a nice and well-behaved Merton-world. It is far from optimal when things are getting non-(log)normal (i.e. more realistic!).
My question
My question is simply: Why is this so? What are the reasons and is there some intuition for this fact.
The Kelly Criterion was derived for two-outcome events (binomial). Assuming it "works" for anything else (including "normal" events) is asking for trouble.
I hope this help you. We have to start from the very first step, namely how the Kelly formula is calculated. We have the chance to make a bet on a event $A$ that as an odd (decimal odds) $O_A$. We want bet only a fraction $f$ of our capital $V_0$. How much of our capital we have to bet? Well, if we win will face with a capital $V_1$ $$ V_1=(1+(O_A-1)f)V_0 $$ If we loose we face with a capital $$ V_1=(1-f)V_0 $$ We now imagine that we can bet again and again in the same event. After $n=w+l$ events, where $w$ are won events and $l$ are lost events, we face with a capital $$ V_n=(1+(O_A-1)f)^w (1-f)^l V_0 $$ We can rearrange the previous equation in this term $$ G_n=\frac{V_n}{V_0}=(1+(O_A-1)f)^w (1-f)^l $$ We want calculate the the unitary growth $G_n$ and we perform a geometric mean applying the $1/n$ roots on both members
$$ G_n^{\frac{1}{n}}=(1+(O_A-1)f)^{\frac{w}{n}} (1-f)^{\frac{l}{n}} $$
Here it is the first assumption. If the two events $w$ and $l$ are not correlated, then we can apply the theorem of large numbers and state that for a large number of trials we must have $$ \lim_{n\rightarrow +\infty} \frac{w}{n}=p \\ \lim_{n\rightarrow +\infty} \frac{l}{n}=q=(1-p) $$ So we have $$ \hat G=(1+(O_A-1)f)^p (1-f)^q $$ A very interesting form is obtained if we apply the $log()$ on both side $$ \log \hat G=\log ((1+(O_A-1)f)^p (1-f)^q)=p\log(1+(O_A-1)f)+q\log(1-f) $$ and interpreting $(O_A-1)=r_p$ and $-1=r_q$ $$ \log \hat G=p\log(1+r_pf)+q\log(1+r_qf)=\sum_i p_i\log(1+r_i f)=E[\log(1+r f)] $$ where $E[\circ]$ is the expected value operator. The Kelly criterion say that the value of $f$ we have to choose is that of maximize $\log \hat G$.
At this point we have made only one assumption, namely that theorem of large number works. The problem arise when we want to apply the previous formula in the continuum limit. In the limit of infinite possible outcomes $i\rightarrow +\infty$ we have $$ \lim_{i\rightarrow +\infty}\sum_i p_i\log(1+r_i f)=\int f(r)\log(1+r_i f)d r= E[\log(1+r f)] $$ The only request in order to calculate the Kelly value of $f$ is that the $ E[\log(1+r f)]$ exists. This impose some restriction in the the convergence of $f(r)$ at infinity. If a $X$ is log-normal distributed variable, this means that must exist the first $\log$ momentum and the second $\log^2$ momentum, namely $$ E[\log X]<+\infty\\ E[\log^2 X] <+\infty $$ So if the return $r$ does not satisfy the first condition, then the value of $f$ we calculate from the Kelly criterion makes no sense and that is the why we can't apply Kelly criterion to the whole universe of distributions.
It helps to think about what the Kelly criterion is attempting to achieve. The purpose of the Kelly criterion is to find a betting strategy that maximizes the geometric growth rate. In a portfolio management context where the investment universe contains a risk-free asset, it would be equivalent to (ignoring constraints) $$ w\equiv argmax\left\{ median\left(\mu_{p}\right)\right\} $$ where $\mu_{p}$ is the arithmetic return of the portfolio over the horizon and $w$ is a vector of weights.
When one adds the assumption that all security prices are log-normally distributed, people will often say that the above is equivalent to $$ w\equiv argmax\left\{ w'm-\frac{1}{2}w'Vw\right\} $$ where $m$ is the mean log return and $V$ is the log covariance matrix. This is typically considered to be equivalent to the mean-variance optimization with a risk aversion coefficient equivalent to 1. While it is true in the univariate case that by accounting for the formula for the median of the log normal distribution and how to convert a normal distribution and log distribution a version of this formula would be created, there are aggregation issues when switching from the multivariate normal to multivariate log normal. Also, when extending this to a non-normal case, it is inconvenient even if you assume away the aggregation issue.
Another option is to instead optimize the arithmetic mean-variance problem (which is the second formula, but with the arithmetic mean and covariance matrices replacing the log versions) to construct the efficient frontier. Then, using the discrete points on the frontier, one can calculate the median return given each efficient portfolio (this is easy to do once the portfolio has been transformed into a univariate mean and variance, the problem above is the aggregataion of the whole portfolio). The investor would then select the optimal portfolio as the one with the largest median return. It would also be trivial to add constraints to the mean-variance problem in case the investor does not wish to have too much leverage (equivalent to a fractional Kelly-like solution).
This can easily be extended to the non-normal case. The investor would calculate the efficient frontier as above, but when calculating the median return could use a simulation-based set of returns at the horizon. All that is required is to calculate the median of each portfolio using these returns and then find the portfolio with the largest median. A simple approach to non-normal Kelly.
