I am running a PCA on a set of returns and I would like to cluster the results of the output to group stocks that have similar factor exposures.

However when I run the PCA on the covariance of the returns, the PCA score (values mapped to new plane of PCs) gives me a matrix with dates and the principal components, to cluster on this would cluster on date therefore.

I can cluster on factor coefficients for each stock but then I have found this ignores the variance. For PC1 for instance the variance of loadings is very low compared to PC2 and therefore when clustering using the loadings it simply clusters using mainly PC2 which seem inherently wrong to me?

Or is this still correct and we can assume that because most stocks are loaded in a similar way to the PC1 then the clustering can’t determine much from that PC anyway.

I’m worried that I am missing some of the information from the variance here as PC1 explains 55% of the variance compared with PC2 at 18%!


A classic problem, been there, done that, didn't buy the T-shirt ;-)

PCA and clustering (K-means, or hierarchical) are similar but different. They're both "unsupervised learning" methods; but one is essentially descriptive, while the other is essentially pragmatic and expedient. People want both, but they need to prioritise one first!

Your PC1/PC2 phenomenon actually makes a lot of sense for stocks. PC1 is beta; and the factor loadings here will indeed be tight - most stocks in the long-run have betas between ~0.8 and ~1.25. Imagine, for simplicity sake's, that your benchmark was dominated by domestic banks and exporting oilcos... Your foreign/domestic/FX/dollar PC2 would indeed generate a much wider set of loadings, even if this effect was much less significant in scale than basic beta (you 18% vs 55% of variance point). And clustering your stocks between your domestics/banks versus your exporters/commodities would, to me, make a lot of intuitive sense.

The caveat - you just have to be comfortable that PC1 is indeed just basic beta, and comfortable that beta is just a factor that all stocks share in common, rather than something that really differentiates them.

Fail that test, and you may need to get your hands dirty with hierarchical clustering. The method bottom-up merges the most similar stocks (or groups of stocks). So it starts off with the easy merging of your oilies, your miners, your tech, banks, industrials etc. Until it then has to start merging industries. But you will get a structured grouping of lookalikes, with transparency how it got there.

  • $\begingroup$ Thanks so much for your answer! And your suggestion for hierarchical clustering I will definitely try that. It’s not a ridiculous thing to do however, cluster on factor loadings? The other way I guess would be to transpose the return matrix and then have PCs as a linear combination of time, then the scores themselves would be the projection of each stock on the new plane and I could cluster on that but I don’t understand the mechanics of it enough. $\endgroup$ – Simon Nicholls Jan 12 at 11:55
  • $\begingroup$ No it's not ridiculous at all, although there are a couple of implicit judgement calls. First, at what point do think that your PCs stop representing bona fides macro/sector exposures, and start representing noise. And if you weight the distance between PC5 loadings the same as between PC1 loadings, you're effectively saying that all of these "signals" are equally important (to you, given clearly PC1 explains a lot more of the variance in your sample). $\endgroup$ – demully Jan 12 at 16:16
  • $\begingroup$ There is no "right" way to do clustering. There are lots of ways of doing, that can produce similar (or sometimes different results). There are just more of less effective methods for any given sample of data; "effective" representing the perceived usefulness of the conclusions to the user. $\endgroup$ – demully Jan 12 at 16:18
  • $\begingroup$ Again thank you for your response super helpful. Just one last question on the point you made about the weighting being equal for each PC, is it sensible to weight them based on the eigen vector, meaning the PC with the most variance would be the most important when clustering? $\endgroup$ – Simon Nicholls Jan 12 at 18:18
  • $\begingroup$ P.S I think you should have bought the t shirt! $\endgroup$ – Simon Nicholls Jan 12 at 18:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.