# Minimizing variance vs. expected shortfall: distributions where the difference is salient

In portfolio theory in finance, given a set of $$n$$ assets to choose from, one often selects portfolio weights so as to maximize expected return and minimize some measure of risk, e.g. variance or expected shortfall*. If we consider asset returns to be random variables, we are looking for a linear combination, with weights summing up to unity, of random variables that has a high expected value and a low measure of risk. Let us consider just two assets, $$n=2$$, and suppose expected returns of all assets are equal to the same constant, e.g. zero.

Questions:

1. What are some bivariate distributions that allow minimizing variance but retaining large expected shortfall, or vice versa?
2. What are some bivariate distributions that would produce substantially different optimal weights when minimizing variance vs. minimizing expected shortfall?
3. What is characteristic to such distributions? (Realistic examples from finance would be appreciated.)

*$$q\%$$ expected shortfall (a.k.a. expected tail loss or conditional value at risk) is simply the mean of the left tail of the random variable, the tail being cut off at the $$q\%$$ quantile level.

The topic covers both finance and statistics. The question has been previously posted at Cross Validated but received zero answers. I am now reposting it here. While cross posting is not recommended, I will keep the post on Cross Validated in case it might eventually attract an answer there.

• If I understand you correct you seem to be looking for a bivariate distribution with low correlation but high lower tail dependence (at least if you don't allow for short sales). One option: you can always add a common low probability but large negative Bernoulli shock to two uncorrelated returns. – fesman Aug 19 '20 at 9:21
• @fesman, thank you! Are there any well-known distributions that can approximate what you have suggested? They could be formulated by two marginals and a copula or some other way. Also, your comment has inspired another question which I have linked to at the end of the post; perhaps you will have some ideas about it, too. – Richard Hardy Aug 19 '20 at 9:45
• Perhaps you can look at skew-t copula. It has separate parameters governing correlation and tail dependence. – fesman Aug 19 '20 at 10:24
• Suppose the two assets have comparable variances. Asset A returns are normally distributed, Asset B returns are generated by a mixture: 90% chance of normally distributed returns and 10% chance of a return of -5% (i.e. a relatively large negative return). Then I conjecture the Cvar algorithm allocates less weight to Asset B than to Asset A, while the Min Var algorithm allocates to them equally. – noob2 Aug 20 '20 at 6:31
• @noob2, I suppose it depends on the interdependence between A and B (technically, the copula). – Richard Hardy Aug 20 '20 at 7:21

Such calculations quickly get messy even in the bivariate case and are best addressed by simulations. That said, the basic question about the fundamental difference between optimisation using Tail risk versus Variance based risk measures can be illustrated by a straightforward calculation using only the total portfolio return.

Put simply the philosophical and practical difference is that Tail risk measures only focus on the tails while Variance incorporates information from the whole distribution. All other differences then follow from this basic distinction.

## Tail/Non-Tail Decomposition

I think it is completely sufficient to analyse the univariate case. Let $$S$$ denote the total portfolio return (e.g. $$S = wX + (1-w)Y$$ for two assets $$X$$ and $$Y$$ with weight $$0\leq w \leq 1$$).

With the tail probability $$0 and the tail quantile $$s_q$$ ( i.e. $$\mathbb{P}[S) we can distinguish between the tail $$\{ S \leq s_q\}$$ and non-tail $$\{ S > s_q\}$$ regions of $$S$$ using the Bernoulli Variable $$Z = \mathbb{1}_{\{ S \leq s_q\}}$$. Let $$F_S$$ be the distribution of $$S$$ and $$\hat{F} = F_S \mid \{Z = 0\}$$ be the upper or non-tail conditional distribution and $$\check{F} = F_S \mid \{Z = 1\}$$ be the lower, tail conditional distribution. Those distributions are lower respectively upper truncated distributions. Furthermore, we need $$\hat{e}$$ and $$\check{e}$$ the expectations as well as the variances $$\hat{v}^2$$ and $$\check{v}^2$$ of $$\hat{F}$$ and $$\check{F}$$.

For simplicity assume that $$S$$ has a continuous density. Then $$-\check{e}$$ is the expected shortfall of $$S$$. By the law of total expectation using $$\mathbb{E}[S]=0$$ one sees that: $$0 = \mathbb{E}[S] = q \check{e} + (1 - q)\hat{e}$$ or $$\hat{e} = -\frac{q}{1-q}\check{e}.\tag{1}\label{1}$$

In the same way, only now using the law of total variance, we can take apart the Variance of $$S$$: \begin{align}\mathbb{V}ar[S] &= \mathbb{E}[\mathbb{V}ar[S\mid Z]] + \mathbb{V}ar[\mathbb{E}[S\mid Z]\\ &= q \check{v}^2 + (1 - q)\hat{v}^2 + \frac{q}{1-q}\check{e}^2\tag{2}\label{2}. \end{align} For the third term one uses the fact that $$Z$$ is Bernoulli with $$\mathbb{P}[Z=1]=q$$ and the relation $$(\ref{1})$$ between the two possible values of $$\mathbb{E}[S\mid Z].$$

## Interpretation

According to $$(\ref{2})$$ the variance can be decomposed into two "within" variances i.e. tail and non-tail variance and an "in-between" variance arising from the difference in mean between tail and non-tail.

So yes indeed, a large expected shortfall will drive the variance. In that sense optimisation of variance and expected shortfall will provide one with similar directions. But the variance incorporates additional terms, which are completely ignored by expected shortfall optimisation. And while arguably and in practice often $$\check{v}^2$$ will be closely related to $$\check{e}$$ by the tails of the available asset distributions, the behaviour of $$\hat{v}^2$$ is often quite separate and somewhat dominant, especially if $$q$$ is very small. Under Variance optimisation it makes a lot of sense to take some more tail risk to get rid of non-tail volatility.

This myopic behaviour is also the reason why pure expected shortfall (or value at risk) optimisation will be rare in practice. It is no consolation to be well-managed on a 1-in-100 years level, if you regularly incur losses.

• Today I finally sat down and read your answer carefully. The decomposition you apply is helpful and illuminating. The notation is a little unusual; I would have used $\alpha$ in place of $q$ and $q_\alpha$ in place of $s_q$, but this is a small thing. Unfortunately, the focus is on the univariate case. It would be very nice to see some bivariate examples. +1 anyway. – Richard Hardy Oct 1 '20 at 14:23
• My point is that there is actually no bi- or multivariate case only properties of the total return distribution. That said, you can check this out yourself easily. Just simulate a portfolio of call options, say, on a single/a few underlying asset with different strike levels. The options with strikes beyond your q-level will constitute the tail all others the body. – g g Oct 1 '20 at 18:40