Portfolio Theory: Why is so much effort put into the reduction of estimation errors?

Question

In MPT, very much effort by researchers is put into developing methods and techniques to handle the rather poor performance of the estimated means, variances and covariances. There are shrinkage estimations, random matrix theory or approaches that address the optimization constraints, for example. Some of them are very complex and sophisticated and in turn require excessive data processing.

At the same time, all these estimation problems arise from the assumption that asset returns are i.i.d. and moreover normally distributed. It has been shown numerous times that these assumptions are contradicted by empirical evidence. Researchers do not seem to agree on how to classify the probability distribution of asset returns. Some mention the classification as an $\alpha$-stable distribution other than the normal distribution, others suppose to classify asset returns as some non-normal distribution with finite second moments.

My question is: Why don't researchers focus on developing a more adequate distribution of asset returns? I mean, handling the estimation errors seems like approaching the problem from the wrong side to me. Wouldn't it be more promising to find an adequate probability distribution of asset returns and to develop models that implement this distribution? Instead of correcting the errors, one would be able to avoid them from the very beginning. I hope I'm not overlooking something. But from what I've seen so far, the focus really seems to be on estimation error reduction.

I'm just asking out of my own interest, because I've been wondering about this. It's quite astonishing to see the above mentioned assumptions being applied although they have been proven to be inadequate.

Thanks a lot in advance and have a good start into the new year.

Answer 1

In MPT investors maximize ex ante expected return for a given level of ex ante variance. Gaussian-ity or iid-ness of returns are not requirements.

The problem is estimating these ex-ante quantities using ex-post transformations of the recent history. Many of the sophisticated techniques that you mention try to offer robustness of ex post estimators in the presence of fat tails and/or lack of iid-ness.

Researchers have correctly (imo) concluded that trying to identify an ever changing 10000-dimensional distribution is futile, therefore it makes sense to try and focus on the quantities of interest only.

Answer 2

There are two ways to answer your question. One is direct and has less depth to it, the other is more indirect and has a lot of depth to it. I will begin with the indirect one because it has wide ranging applications beyond finance or economics and because it should serve as a warning to journal editors and so forth. Also, it covers an area of statistics that everyone but statisticians have forgotten about.

Although the idea of a statistic is rather old, the field of statistics is rather new. It is probably the newest or nearly newest of all fields. Aeronautics is older. Genetics is older. It opened up a ton of practical questions that it took time to solve. It has to do with how the field defined as statistic. A statistic is any function of the data. This means that almost every statistic is useless as there are uncountably many functions.

This led to a process to decide which statistics to keep and which to discard. This created unexpected results. If you are finding poor estimators as a result of your theory, then there is a good chance, you are doing it wrong. In the defense of finance, it has been struggling with this since Mandelbrot published the first empirical refutation of mean-variance finance. It tried to solve it in the 1960's but a couple of things got in the way. The first was the use of punch card technology. Even if work by Eugene Fama or Mandelbrot were correct, it would have resulted in problems that would take decades to solve. The second was that there was no reason for them to be correct. There was no theory behind the observations.

The unexpected result, in searching for a statistic, was that all Bayesian statistics were admissible. This was surprising because it was proved with Frequentist axioms. It also found that all other statistics were valid to the extent they either mapped to a Bayesian measure in a particular case or at the limit. It provides a test, however. If you can stochastically dominate a measure, then you drop that measure. If you hunting for accurate measurements that work, then something deeper is going on and you are missing it.

The more direct answer is that the distribution of returns, for the Markowtiz model to be correct, have to have certain properties. The first is that there needs to be a mean in order to have an expectation in the first place. Most standard distributions have a mean, but not all do. The Cauchy distribution and, in general, the Paretian distributions of Mandelbrot's article do not. The Cauchy distribution is $$\frac{1}{\pi}\frac{\sigma}{\sigma^2+(x-\mu)^2}.$$

The second is that if a mean exists, a covariance matrix needs to exist. Not all distributions with a variance have a covariance in its multivariate form. The hyperbolic secant distribution is an example of that. It is $$sech\left(\frac{x-\mu}{\sigma}\right).$$

There have been attempts to use both in empirical finance. If either of those distributions are present in the likelihood function, then mean-variance finance is indefensible. The former is problematic because you cannot form an expectation on your returns in the first place. They are excluded by the laws of general summation. The second is a bit more subtle because if the second is present none of the assets can be independent, but none of them can covary either. They can comove, but not covary. It creates a very ugly issue.

There is a paper that derives the distribution of returns at https://ssrn.com/abstract=2828744. It shows that there are many distributions that can be present. The logic of the paper is that returns are not data, rather, prices are data. Returns are transformations of data. In particular, they are the ratio of jointly distributed variables, a present value and a future value. The distribution depends upon the rules in use to create the prices. As a result, stocks have different returns than antiques because the auction process is different.

As it happens, all distributions for equity securities include some mixture of a transformation of the Cauchy distribution. Because the distributions involved lack a sufficient statistic, any point estimator has to lose information, so no non-Bayesian solution exists for projective problems (such as choosing an allocation), and should be avoided for inferential questions if possible. You cannot avoid them in your true hypothesis is a sharp null hypothesis as there is no good Bayesian solution for sharp null hypotheses.

A population test of the paper can be found at https://ssrn.com/abstract=2653151

There are also papers to replace the method of pricing options and the rules of econometrics. Papers to create optimal portfolios and to extend stochastic calculus are in process. The distributions paper will be presented at the Southerwestern Finance Association Conference in March.

Some things will have to change. You cannot make an assumption of i.i.d. variables, for example. The entire discussion of the Solow convergence will have to change in economics and so the core of the whole discussion of capital, physical, financial and human.

A lot of focus will end up on the scale parameter. In the Cauchy distribution, there is no covariance matrix. If you had a one asset portfolio, denoted $a$, then it may have a scale parameter $\gamma_a$. If you switch to a two asset portfolio you do not get two scale parameters, let alone a covariance style matrix. Instead you get a new scale parameter $\gamma_{ab}$. If you got fancy and used a vector process, all the vectors would jointly share a scale parameter $\gamma_v$. Taking the logarithm brings you to the hyperbolic secant distribution and so no gain is had. It also has no covariance matrix, but OLS does. OLS would be measuring something that does not exist.

The headaches are just starting.

Answer 3

Just to provide some background on why mean-variance is so sensitive to estimation error, consider an unconstrained mean-variance optimization. The optimal weights are given by $$w=\frac{1}{\lambda}\Sigma^{-1}\mu $$ where $\lambda$ is a risk aversion coefficient, $\Sigma$ is the covariance matrix, and $\mu$ is a vector of returns.

We can examine the sensitivity of the weights with respect to the parameters, by calculating the partial derivatives. The most relevant is $$\frac{\partial w}{\partial\mu}=\frac{1}{\lambda}\Sigma^{-1} $$ The practical nature of the covariance matrix is that it will typically have very high values when inverted. As a result, this means that a small change in $\mu_{i}$ will have a large impact on all $w$. One can do the same analysis with variances and covariances, but it is less of an issue.

The problem of estimation error is that classical mean-variance optimization is very sensitive to return forecasts.

The question then is why focus so much on estimation error when the problem is the return models. In some sense, shrinkage models and random matrix theory are actually an attempt to create better models. They are creating models in recognition that classical mean-variance is susceptible to estimation error.

The question suggests non-normal models. While there is value in non-normal models, they are not a solution for estimation error. If the techniques that reduce the impact of estimation error rely too heavily on normal distributions, then tweak the assumptions and see if you can make them work with non-normal distributions.

Most importantly: if you have a hammer, try not to hammer your finger.