PCA and K-means clustering on returns

Question

I am running a PCA on a set of returns and I would like to cluster the results of the output to group stocks that have similar factor exposures.

However when I run the PCA on the covariance of the returns, the PCA score (values mapped to new plane of PCs) gives me a matrix with dates and the principal components, to cluster on this would cluster on date therefore.

I can cluster on factor coefficients for each stock but then I have found this ignores the variance. For PC1 for instance the variance of loadings is very low compared to PC2 and therefore when clustering using the loadings it simply clusters using mainly PC2 which seem inherently wrong to me?

Or is this still correct and we can assume that because most stocks are loaded in a similar way to the PC1 then the clustering can’t determine much from that PC anyway.

I’m worried that I am missing some of the information from the variance here as PC1 explains 55% of the variance compared with PC2 at 18%!

Answer 1

A classic problem, been there, done that, didn't buy the T-shirt ;-)

PCA and clustering (K-means, or hierarchical) are similar but different. They're both "unsupervised learning" methods; but one is essentially descriptive, while the other is essentially pragmatic and expedient. People want both, but they need to prioritise one first!

Your PC1/PC2 phenomenon actually makes a lot of sense for stocks. PC1 is beta; and the factor loadings here will indeed be tight - most stocks in the long-run have betas between ~0.8 and ~1.25. Imagine, for simplicity sake's, that your benchmark was dominated by domestic banks and exporting oilcos... Your foreign/domestic/FX/dollar PC2 would indeed generate a much wider set of loadings, even if this effect was much less significant in scale than basic beta (you 18% vs 55% of variance point). And clustering your stocks between your domestics/banks versus your exporters/commodities would, to me, make a lot of intuitive sense.

The caveat - you just have to be comfortable that PC1 is indeed just basic beta, and comfortable that beta is just a factor that all stocks share in common, rather than something that really differentiates them.

Fail that test, and you may need to get your hands dirty with hierarchical clustering. The method bottom-up merges the most similar stocks (or groups of stocks). So it starts off with the easy merging of your oilies, your miners, your tech, banks, industrials etc. Until it then has to start merging industries. But you will get a structured grouping of lookalikes, with transparency how it got there.