48
$\begingroup$

I have a sample covariance matrix of S&P 500 security returns where the smallest k-th eigenvalues are negative and quite small (reflecting noise and some high correlations in the matrix).

I am performing some operations on the covariance matrix and this matrix must be positive definite. What is the best way to "fix" the covariance matrix? (For what it's worth, I intend to take the inverse of the covariance matrix.)

One approach proposed by Rebonato (1999) is to decompose the covariance matrix into its eigenvectors and eigenvalues, set the negative eigenvalues to 0 or (0+epsilon), and then rebuild the covariance matrix. The issue I have with this method is that:

  1. the trace of the original matrix is not preserved, and

  2. the method ignores the idea of level repulsion in random matrices (i.e. that eigenvalues are not close to each other).

Higham (2001) uses an optimization procedure to find the nearest correlation matrix that is positive semi-definite. Grubisic and Pietersz (2003) have a geometric method they claim outperforms the Higham technique. Incidentally, some more recent twists on Rebonato's paper are Kercheval (2009) and Rapisardo (2006) who build off of Rebonato with a geometric approach.

A critical point is that the resulting matrix may not be singular (which can be the case when using optimization methods).

What is the best way to transform a covariance matrix into a positive definite covariance matrix?

UPDATE: Perhaps another angle of attack is to test whether a security is linearly dependent on a combination of securities and removing the offender.

Improve this question
$\endgroup$
1
  • 7
    $\begingroup$ Since negative eigenvalues imply imaginary variances, do you really require the trace to be preserved? In a financial setting, I have yet to encounter a scenario where the difference between the truly Frobenius-norm nearest positive semi-definite matrix and one generated from spectral truncation was large enough to warrant the additional effort. $\endgroup$
    – michaelv2
    Commented Oct 2, 2011 at 12:10

4 Answers 4

Reset to default
22
$\begingroup$

Nick Higham's specialty is algorithms to find the nearest correlation matrix. His older work involved increased performance (in order-of-convergence terms) of techniques that successively projected a nearly-positive-semi-definite matrix onto the positive semidefinite space.

Perhaps even more interesting, from the practitioner point of view, is his extension to the case of correlation matrices with factor model structures. The best place to look for this work is probably the PhD thesis paper by his doctoral student Ruediger Borsdorf.

Higham's blog entry covers his work up to 2013 pretty well.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Great find. Higham's talk is based on a recent paper, Computing a Nearest Correlation Matrix with Factor Structure. This fact is a bit difficult to find within the talk, which covers a lot of other areas, too. $\endgroup$
    – Tal Fishman
    Commented Oct 3, 2011 at 21:12
  • $\begingroup$ The link appears to be dead. $\endgroup$
    – Mack
    Commented Nov 29, 2015 at 22:19
8
$\begingroup$

In Oracle Crystal Ball, we use an old algorithm, that works pretty well and converges fast. It is from Iman-Conovar. Here is the reference:

Iman, R.L., Conover, W.J. 1982. A distribution-free approach to inducing rank correlation among input variables. Commun. Statist.-Simula. Computa. 11, 311-334.

That said, Prof. Higham's method based on optimization works pretty good as well. He seems to have updated methods that were originally presented by Lurie and Goldberg, available at this link.

Improve this answer
$\endgroup$
0
7
$\begingroup$

The short answer is that I don't know, but your question gives some hints about how to find out. The key thing for me is that you want a minimum variance portfolio. I don't think you should be thinking about some abstract mathematical operation that is "best", but rather look over a few mathematical operations and see which seems to work best for your application.

If you can approximate the problem you have now with data in the past, then you can test different methods over time.

Some observations:

  • if you have a long-only portfolio, then that already helps you out a lot.

  • If you have lots of assets relative to time points, then it is harder: you want to use (if possible) Ledoit-Wolf shrinkage or a factor model to estimate the variance matrix.

  • You don't want the optimizer to see any portfolios that have zero (or negative) risk. That is the finance version of positive eigenvalues -- you want your epsilon to be non-trivial.

  • You don't say why you start out with a non-positive definite matrix. If it is because of missing values and you have the original returns, then there is code to do Ledoit-Wolf shrinkage in such a case.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ I am actually testing which random-matrix theory cleansing methods and covariance shrinkage methods (constant correlation, etc.) produce the most stable covariance matrix (as measured by the out-of-sample performance of the minVar portfolio). The way I am doing this is by finding the weights of the cleansed/shrunk matrix corresponding to a minimum variance portfolio, and then measuring the variance of this portfolio out-of-sample. $\endgroup$
    – Ram Ahluwalia
    Commented Oct 1, 2011 at 17:06
  • $\begingroup$ FYI - I updated the question and dropped the reference to minimum variance and focused on inversion of the matrix itself $\endgroup$
    – Ram Ahluwalia
    Commented Oct 1, 2011 at 20:39
  • $\begingroup$ Also, I updated the source of the covariance matrix. The negative eigenvalues result from the sample covariance matrix itself. I provided an upvote particularly because of the insight of your 3rd bullet point $\endgroup$
    – Ram Ahluwalia
    Commented Oct 1, 2011 at 20:41
2
$\begingroup$

Here it is: "Rebonato, R., Jackel, P. The most general methodology to create a valid correlation matrix for risk management and option pricing purposes."

Recall: a covariance matrix will be the same as a correlation matrix if scale is removed. I used this method for ensuring positive definite correlations matrices.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.