I'm trying to do PCA on historic forward rates. I'm using forward rates from the Bank of England going from Jan 2015 through end of May 2018. I calculate the differences in the rates from one day to the next and make a covariance matrix from these difference. The matrix is 51 x 51 (because the tenors are every 6 months to 25 years plus a 1 month tenor at the beginning). My matrix is not positive definite which is a problem for PCA. I don't understand why it wouldn't be. The data is "clean" (no gaps).
$\begingroup$
$\endgroup$
6
-
1$\begingroup$ So you have $N=51$ variables. How many observations $T$ do you have from Jan2015 to May2018. Is $T<=N$ or $T>N$ ? $\endgroup$– Alex CCommented Jul 3, 2018 at 20:40
-
1$\begingroup$ What's the smallest eigenvalue of the covariance matrix? (If $T > N$ you'll have at least $T-N$ numerically zero eigenvalues.) $\endgroup$– Matthew GunnCommented Jul 3, 2018 at 20:57
-
$\begingroup$ T>N because the time series is daily going back to Jan 4 2015 and ending May 31 2018 $\endgroup$– M ThomasCommented Jul 3, 2018 at 22:42
-
$\begingroup$ It has some negative eigenvalues which is part of the problem. None of the eigenvalues are zero $\endgroup$– M ThomasCommented Jul 3, 2018 at 22:43
-
3$\begingroup$ the short answer, is because it's not a covariance matrix. How did you calculate it? $\endgroup$– willCommented Jul 4, 2018 at 6:32
|
Show 1 more comment
2 Answers
$\begingroup$
$\endgroup$
'Not positive definite' is an algebraic statement that some of the variables are linear combinations of one another. The problem then becomes one of tracking down the offending variates.
I've used two brute-force approaches for this but neither scales well in the presence of large amounts of information. One method is to examine pairwise correlations and partial correlations looking for very high r-values, e.g., r>=0.95. A second tactic is much more nitty-gritty and involves scrutinizing the variable-level scores across the resulting components as output from the PCA. By sorting the variables on their first few components one can identify variables with the same or highly similar score values.
I'm sure other QF participants have much more sophisticated tactics that do scale well to large data.
$\begingroup$
$\endgroup$
1
You have not shown data, so one can only guess.
If you have computed the covariance matrix from the full dataset with no missing values (and you have not used some weird estimator), then the only way to have negative eigenvalues is round-off error: in that case, those negative eigenvalues will be practically zero: so just replace them with zero. See http://comisef.wikidot.com/tutorial:repairingcorrelation.
A different question is whether your covariance matrix has full rank (i.e. is definite, not just semidefinite). If you have at least n+1 observations, then the covariance matrix will inherit the rank of your original data matrix (mathematically, at least; numerically, the rank of the covariance matrix may be reduced because of round-off error). So you should check your original data matrix, whether it has rank 51, or less.
