# Weights Blowing up in PCA

I'm using daily settlement data to get yield levels for a couple of products. From this data I am doing PCA on a rolling collection of the yield levels. I have been using sci-kit learn's PCA function, but I also see the issue when doing my own PCA through Numpy. So as far as I know it's not an issue of the libraries.

After I get the vectors I solve the linear equations such that the first two principle components sum to 0. This is done by setting one of the weights = 1.0

Here's an example. I have data for 150 settlements and I calculate the PC's using data from day 0-100, then I recalculate 10 days later on data 10-110, etc.

When I do this I get a graph of the PC's And here are the corresponding weights. Relevant math: After performing the PCA I get the components matrix $~ \left( \begin{array}{ccc} a & b & c \\ d & e & f \\ g & h & i \end{array} \right)$ From here I take the first two vectors $[a, b, c]$ and $[d, e, f]$. Which I then turn into the equation $Ax = B$ that looks something like this $~ \left( \begin{array}{ccc} a & c \\ d & f \end{array} \right) \left(\begin{array}{ccc} x_1 \\ x_3 \end{array} \right) = \left(\begin{array}{ccc} b \\ e \end{array} \right)$

As you can see the $x_1$, $x_3$ weights start to blow up at some point which doesn't really make sense given the nature of the data.

Does anybody have any insight to my problem?

• When you say "a couple" do you actually mean 2? Your second and third components look to be extremely highly (negatively) correlated, suggesting you could have some sort of linear dependence. – will May 13 '17 at 10:58

## 1 Answer

A couple quick thoughts.

1. Do the PCA on changes or log-changes in your series. That is often how PCA is conducted in fixed-income settings.
2. You're large move in wights corresponds to outlier moves in the blue series. Given the assumptions of a PCA, I would consider whether your dataset has suffered from any breakpoints, regime changes or other rare events
3. Think about smoothing your weights (with some priors). Remember that you are trying to explain the interaction of three fixed income instruments (which are driven by economies, politics, and market forces) by only 9 parameters. You need to match your analysis with reasonable expectations on its performance.
• Agree with 2 & 3. But lots of people use PCA on levels for RV purposes. – Helin Aug 22 '15 at 8:18
• @closedloop I have tried the PCA on changes/log-changes before this post, I probably should have mentioned that. I'm doing it on levels because, for the most part, yields are already normalized and it makes more sense for what I am trying to model. As for 2, these are future contracts and I'm guessing my rolling algorithm might be causing some strange discontinuities that I will have to investigate. 3. Do you have any literature on weight smoothing in a PCA framework that you would recommend? – meh Aug 25 '15 at 13:50
• @meh I'd look at the papers describing Black-Litterman or anything on Bayesian PCA. – closedloop Aug 25 '15 at 13:54