When using Markowitz Portfolio Theory, e.g. for finding an optimal portfolio composition, one needs to have estimates of the returns, but most importantly of the covariance matrix. If our universe of assets/securities was, say, larger than 10,000 names (or just a very large number), how would one effectively come up with a useable estimate for the covariance matrix?

Clearly, we can use historically-observed covariance/correlations but given that we have such a large number of names, how accurate would these estimates be? Is it possible to reduce dimensionality by using some sort of PCA approach?

I am not interested in a perfect solution, but rather in ideas and potential techniques that I could read up on w.r.t. portfolio optimisation and parameter estimation on a large scale.

  • $\begingroup$ A good paper for the Factor (or what you call PCA) approach is Journal of Financial Economics 62 (2001) 293–325, Extracting factors from heteroskedastic asset returns, by Christopher S. Jones $\endgroup$
    – Alex C
    Jun 4, 2017 at 20:26
  • $\begingroup$ @AlexC again, please... Let's put this in an answer. $\endgroup$
    – SRKX
    Jun 7, 2017 at 1:30

1 Answer 1


Broadly speaking, as you probably already know, there are 2 approaches to estimating large covariance matrices:

1) Shrinkage Methods like Ledoit-Wolf that try to reduce the noise in a large matrix (N by N) that has been estimated using the conventional method.

2) Factor Models of Covariance as described in for example Connor Korajczik 2007 that assume that only a small number of factors matter. This is pretty much what you have in mind when you mention "some sort of a PCA approach". A decent paper on estimating such models is Extracting factors from heteroskedastic asset returns, by Christopher S. Jones

On the practical side there is a big project at Blackrock directed by Andrew Ang that is developing and updating a huge factor-based risk model for a large number of markets that is the biggest covariance estimation that I know of. Bloomberg is also developing a large risk model of this kind.

There are a lot of practical issues that you have to deal with if you do the estimation yourself. One is that not all assets have been in existence for the same length of time (Stambaugh wrote about this in 'Analyzing Investments Whose Histories Differ in Length' in 1997). Another is that daily correlations between stocks trading on different exchanges are tricky because, for example the trading hours for New York and Tokyo are quite different. Also, because of these and other problems the covariance matrix you estimate often turns out to be non-positive definite, which causes problems when you run an optimizer. You have to check the matrix before you use it and fix it to be positive definite if necessary.

  • 1
    $\begingroup$ Stambaugh 97 is highly recommended, very practical $\endgroup$
    – user25064
    Jun 7, 2017 at 12:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.