I have a 51 x 51 covariance matrix that is derived from historic forward rates that is positive definite. I know it is because in Python np.cholesky returns a correct cholesky decomposition. However, when I use the eigh function to return the eigenvalues/vectors, about half of the eigenvalues are negative, which makes no sense. The data is imported from excel using pandas dataframes, then I make a covariance matrix using:

CovMat2 = np.cov(df.T)*252/10000

Then I request the eigenvalues with

eig_vals, eig_vecs = np.linalg.eigh(CovMat2)
print('Eigenvectors \n%s' %eig_vecs)
print('\nEigenvalues \n%s' %eig_vals)
and I get this output:

[-1.35963506e-03 -1.09487110e-03 -5.00507744e-04 -4.76180253e-04
-4.26481890e-04 -3.65765795e-04 -3.21245207e-04 -2.90812021e-04
-2.49174935e-04 -2.47519558e-04 -2.20946393e-04 -2.11019230e-04
-2.01940366e-04 -1.99581473e-04 -1.87201389e-04 -1.74938799e-04
-1.41441186e-04 -1.24247169e-04 -1.11167101e-04 -9.78191552e-05
-8.75246993e-05 -8.69367038e-05 -8.36134890e-05 -8.03187245e-05
-7.36730798e-05 -5.70815773e-05 -4.65981850e-05 -2.92229574e-05
-2.77958421e-05 -1.84175309e-05 -7.45464337e-06  1.37165843e-05
 2.91872261e-05  3.93959482e-05  4.46749803e-05  6.07783927e-05
 7.42504887e-05  7.99764211e-05  9.49821563e-05  1.05894570e-04
 1.15379642e-04  1.37420008e-04  1.47812090e-04  2.07834453e-04
 2.63958555e-04  2.75403037e-04  3.01136833e-04  3.06428511e-04
 3.75549199e-04  1.96296188e-03  5.80951459e-03]

which makes no sense. What am I doing wrong? Perhaps it's something to do with converting dataframes to a numpy matrix?

  • $\begingroup$ Does your covariance matrix look as you expect it to? Can you share a snippet? $\endgroup$ – Scott Skiles Jul 7 '18 at 15:40
  • $\begingroup$ The values in the covariance matrix are small because it's calculated on the differences in interest rates in a time series but yes I'm positive that it's correct $\endgroup$ – M Thomas Jul 7 '18 at 16:36
  • $\begingroup$ Eigenvectors can be negative, much of your post mentions negative eigenvalues but in your text you specifically say eigenvectors (a mistype I think). If np.cholesky returns a decomposition, and your matrix is truly symmetric, then the only remaining option for negative eigenvals is numerical precision. Is your dtype 'float64'. $\endgroup$ – Attack68 Jul 7 '18 at 19:34
  • $\begingroup$ You're right, that was a typo, I meant the eigenvalues are negative. I haven't specified the data type, maybe I'll try that $\endgroup$ – M Thomas Jul 7 '18 at 22:19
  • $\begingroup$ datatype is float64 $\endgroup$ – M Thomas Jul 7 '18 at 23:27

It appears that changing the dataframe to values sorts this problem out. df = df.values before calculating the covariance matrix gives correct eigenvalues.

  • $\begingroup$ Yikes! The kind of trap that makes me distrust the R language. Probably the data was being represented as a 'factor' or 'enum' rather than actual values. $\endgroup$ – Alex C Jul 8 '18 at 17:30
  • 1
    $\begingroup$ @AlexC The code is clearly in Python/numpy, not R. $\endgroup$ – steveo'america Jul 9 '18 at 20:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.