What methods do you use to improve expected return estimates when constructing a portfolio in a mean-variance framework?

Question

One of the main problems when trying to apply mean-variance portfolio optimization in practice is its high input sensitivity. As can be seen in (Chopra, 1993) using historical values to estimate returns expected in the future is a no-go, as the whole process tends to become error maximization rather than portfolio optimization.

The primary emphasis should be on obtaining superior estimates of means, followed by good estimates of variances.

In that case, what techniques do you use to improve those estimates? Numerous methods can be found in the literature, but I'm interested in what's more widely adopted from a practical standpoint.

Are there some popular approaches being used in the industry other than Black-Litterman model?

---

Reference:

Chopra, V. K. & Ziemba, W. T. The Effect of Errors in Means, Variances, and Covariances on Optimal Portfolio Choice. Journal of Portfolio Management, 19: 6-11, 1993.

Answer 1

Short of having a 'reasonable' predictive model for expected returns and the covariance matrix, there are a couple lines of attack.

1. Shrinkage estimators (via Bayesian inference or Stein-class of estimators)
2. Robust portfolio optimization
3. Michaud's Resampled Efficient Frontier
4. Imposing norm constraints on portfolio weights

Naively, shrinkage methods 'shrink' (of course,no?) your estimates (arrived at using historical data), toward some global mean or some target. Within the mean-variance framework, you can use the shrinkage estimators, for both, the expected returns vector, as well as the covariance matrix. Jorion introduced application of a 'Bayes-Stein estimator' to portfolio analysis. Bradley & Efron have a paper on the James-Stein estimator. Alternatively, you can stick to the global minimum variance portfolio, which is less susceptible to estimation errors (in expected returns)), and use either the sample covariance matrix or a shrunk estimate.

Robust portfolio optimization seems to be another way 'nicer' portfolios can be constructed. I haven't studied this in any detail, but there's a paper by Goldfarb & Iyengar.

Michaud's Resampled Efficient Frontier is an application of Monte Carlo and bootstrap to addressing the uncertainty in the estimates. It is a way of 'averaging' out the frontier and it perhaps is best to read up Michaud's book or paper to know what they really have to say.

Finally, there might be a way to directly impose constraints on the norm of the portfolio weight vector which would be equivalent to regularization in the statistical sense.

Having said all that, having a good predictive model for E[r] and Sigma, is perhaps worth the effort.

References:

Jorion, Philippe, "Bayes-Stein Estimation for Portfolio Analysis", Journal of Financial and Quantitative Analysis, Vol. 21, No. 3, (September 1986), pp. 279-292.

Philippe Jorion, "Bayesian and CAPM estimators of the means: Implications for portfolio selection", Journal of Banking & Finance, Volume 15, Issue 3, June 1991

Robert R. Grauer and Nils H. Hakansson, "Stein and CAPM estimators of the means in asset allocation", International Review of Financial Analysis, Volume 4, Issue 1, 1995, Pages 35-66

Donald Goldfarb, Garud Iyengar: "Robust Portfolio Selection Problems". Math. Oper. Res. 28(1): 1-38 (2003)

Michaud, R. (1998). Efficient Assset Management: A Practial Guide to Stock Portfolio Optimization, Oxford University Press.

Answer 2

Both answers from Shane and Vishal Belsare make sense and detail different models. In my experience, I have never been satisfied by a unique model since the majority of papers out there can be split in two categories:

1. Those that predict the mean component of the problem.
2. Those that predict the variance component of the problem.

The ideal (to read "practical") model would be the one that allows you to incorporate your own views in both expectation of returns and variance.

On the expected returns, Black-Litterman seems interesting since it enables you to get a relative point of view of the expectations which is far more stable and less risky than absolute expected returns. On the variance side, you can use two variance matrices. Theoritically, this would be using a markov switch regime regression or a 2-state regression. There is enough literature on the markov switch model that you can read, the latter model is simpler and easier to use. It consists in considering the returns of your assets as a bivariate normal distribution, one that explains the returns in quiet state of the market and the other explains the hectic state in the market. The result of such regression would be a variance matrix conditioned by the state of the market. (You can use, then, the VIX as a proxy of the state of the market in order to choose between both).

I have tried in the past different models, but, to my opinion, this framework seems to be ahead of the theoretical ones.

I'll add some references that may be of interest:

Kim, J. and Finger, C., A Stress Test to Incorporate Correlation Breakdown, Journal of Risk, Spring 2000

McLachlan, G. and Basford, K., Mixture models: Inference and Applications to Clustering, Marcel Dekker Inc., 1988

Answer 3

You raise a very important point, which unfortunately doesn't have a simple answer.

Black-Litterman addresses the allocation problem by allowing you to provide a prior within a bayesian framework. It doesn't really tell you how to produce the prior itself. But more importantly, it doesn't address the fundamental problem: it's difficult to accurately predict expected returns.

So, you can improve this by having a better model to predict the expect returns besides assuming a static, simple linear model ("this was the mean return over the last $n$ years"). But improving it is the big challenge in finance in general. And standard textbook models haven't done too much to improve the situation; the most success in time series modeling has been around volatility prediction (e.g. with some of the GARCH models), which addresses the variance part of the problem. But ARIMA and other time series models have mixed success success when trying to predict returns for financial assets.

Answer 4

The answer to the original question is simple: the Chopra-Ziemba paper is highly flawed and unreliable. Note that the framework is in-sample and based on a utility function. It has nothing to do with out-of-sample behavior of the mean vs. the covariance in an optimization. Estimation error grows linearly in the mean but quadratically in the covariance. At the portfolio optimization level, unless you are optimizing four or less assets, the estimation error in the covariance begins to dominate. It is one of the classic errors that keeps getting referenced. The Ledoit estimator talks about estimation error in the covariance and offers a solution.

Robust optimization is just a set of constraints on the optimization. It is no solution to the estimation error problem in portfolio optimization. It is robust simply because it is constrained.

The problem with Black-Litterman is that it is not an alternative at all to estimation error. Our newly published paper: "Deconstructing Black-Litterman: How to get the portfolio you already knew you wanted" by Michaud, Esch, Michaud, Journal of Investment Management, 1st quarter 2013 details a wide variety of critical errors in the procedure that is inconsistent with modern statistics and in no way solves estimation error issues.

@QuantGuy continues to propound errors about Michaud optimization. The Michaud efficient frontier has a "maximum return point" depending on the character of the inputs. The Scherer claim of an anomaly is based on a serious error of misunderstanding. When there is insufficient information in investment inputs, things can occur. But they make perfect investment sense. See Michaud and Michaud (2008, ch. 6) for a discussion.

Answer 5

One approach which I've encountered in practice is Optimal risk budgeting (ORB). This method is similar to Black Litterman in the sense that it uses active investor views as a starting point. The mean variance optimization is then restricted to those assets for which an active investor view is available, and the allocation is calculated with the constraint of an overall risk budget (a maximum tracking error or volatility of the portfolio).

W. Lee & D.Y.Lam, 'Implementing Optimal Risk Budgeting', Journal of Portfolio Management 2001 (Vol 28) 73-80.

The ORB approach uses the view confidence of the investor as an estimate of the expected return. In practice, this is typically an expected return spread of an active view such as 'US equities will outperform EU equities by $x$'. The volatility of the view portfolio $r_L - r_S$ is commonly used to estimate this spread.