I'm looking to run portfolio optimizations using various optimization goals - e.g. minimum variance, max diversification etc. My challenge is if I want to do this on ETF's which ones do I pick to run the optimization on?
Say there is a universe of 200 or so ETF's - is there some form of clustering I can do to reduce this down to a smaller set of 20 or so to optimize? Or is this best handled by letting the portfolio optimizer itself apply the appropriate weights out of the larger set?
What techniques should I consider for clustering - i.e. what metrics make sense is it correlation, mean return (I doubt it since that's so noisy), anything else?
To clarify what I'm trying to do is whittle down the 500 names in the SP500 to 20 or so clusters and then from each cluster take the most representative stock to get 20 names. I would then do portfolio opt on the 20 names Following a series from the amazing systematic investor blog I've been able to get really nice results doing some clustering as follows:
# Try various clustering schemes on xrets (log returns matrix) xrets.scaled <- scale(xrets) xrets.euclid.dist <- dist(t(xrets.scaled)) xrets.correlation.distance <- as.dist(1-cor(xrets)) fit <- hclust(xrets.correlation.distance, method="ward") plot(fit) k.numclusts <- 30 groups <- cutree(fit, k=k.numclusts) rect.hclust(fit, k=k.numclusts, border="red")
However all this gives me is clusters e.g. in the SP500 it found clusters of the following form (illustrating just one cluster):
> CAG : ConAgra Foods Inc. :  NA CL : Colgate-Palmolive Co. :  NA > CPB : Campbell Soup Co. :  NA GIS : General Mills Inc. :  NA HNZ > : H. J. Heinz Company :  NA HRL : Hormel Foods Corp. :  NA K : > Kellogg Company :  NA KMB : Kimberly-Clark Corporation :  NA KO > : The Coca-Cola Company :  NA MKC : McCormick & Co. Inc. :  NA > PEP : Pepsico Inc. :  NA PG : Procter & Gamble Co. :  NA SJM : > The J. M. Smucker Company :  NA
This still doesn't tell me though which stock in each cluster is the most optimal to select for further portfolio opt. Any ideas?