Dendrogram/cluster analysis of correlation matrix

Question

First post, hope I'm explaining things sufficiently clearly. I want to take a universe of potential, trade-able instruments and allocate them to portfolio managers. Traditionally, this is done using a sector classification, such as GICS. When PMs submitted their owned proposed universes they obviously had their own sector classifications which differed from GICS which made me think: 1. Is GICS the best way of organizing things (which I've always concluded that it's not ideal but better than all the other methods)? 2. Is there a statistical way to take all the instruments within our universe and do a cluster analysis and make a dendrogram and then compare the clustering to both GICS and the PM's personally chosen subsets? This would also help for instances of overlap of chosen subsets, as it would help to make the decision as to who should get the overlapping instruments. 3. I created a correlation matrix, and converted it to distances using d^2 = 2(1-|r|) I'm trying to figure out the next step and I'm having a few issues. Euclidean distances are comparable for each node calculation, but it seems like each subsequent node needs a new matrix and new distances as the combination of a prior node represents an additional row/column to add to the original matrix. Since distances are not additive due to lack of orthogonality.

Any ideas and code suggestions for doing something in python would be greatly appreciated.

Answer 1

To summarize, you're attempting to create statistical 'sectors' in lieu of more standard equity classifications (eg, GICS, ICB)?

It's something you can do, though to be frank, it's likely a fools-errand. For one, the robustness of your clusters is likely to be pretty weak even if you're thoughtful about the variable(s) you're clustering on and your approach. I'm guessing you're using monthly returns at this point? There's a LOT that drives individual security returns aside from sector-similarity, hence it'll be difficult to get any meaningful results.

Not to mention, actually using the clusters will be a difficult sell. 'Fancy' only really works (unless you're the boss) when its benefits are clearly and unambiguously demonstrated, particularly in the face of industry standards. IMO, I don't think that's possible doing what you're trying to do.

The rest are implementation issues. I'd suggest getting a text that covers clustering or googling. It's not really something someone can 'tell' you how to do absent sitting together and working through it.

HTH