How do I find the most diversified portfolio, or least correlated subset, of stocks?

Question

I have a trading system that chooses top 10 stocks in Nasdaq 100 ranked on relative strength and some other factors. However, I'd like to take positions in only 5 of these 10 stocks based on how minimally correlated these are to others for diversification effect. How do I resolve this? I do have the correlation/covariance matrices computed. Literature seems to indicate applying weights to reduce correlations but I felt there should be a simpler solution. That said, the stocks doesn't need to be equally weighted if it is easier to compute these weights.

A computationally easier solution is preferred even if it is not completely accurate since I need to implement this in Amibroker trading software.

Answer 1

One simple method, based on the principles of mean-variance optimization, is to set the weights proportional to the product of the inverse of the covariance matrix and a vector of standard deviations. This implicitly assumes that the normalized expected return of each stock is equal. If you wish, you can take only the top 5 weights and set the others to zero. The actual problem you face, of selecting just 5 stocks, can be solved rigorously with an optimizer, but since it is not a quadratic program, may be difficult to solve.

Update

A more sophisticated but very interesting additional possibility is to find the "Maximum Diversification Portfolio (MDP)", as defined in Toward Maximum Diversification (free version, hat tip vonjd). The MDP is defined as the portfolio that maximizes the Diversification Ratio (DR), which in turn is defined as the ratio of the portfolio’s weighted average volatility to its overall volatility. A follow-up paper investigates the properties of this portfolio. From the paper:

This measure [DR] embodies the very nature of diversification whereby the volatility of a long-only portfolio of assets is less than or equal to the weighted sum of the assets' volatilities. As such, the DR of a long-only portfolio is greater than or equal to one, and equals unity for a single asset portfolio. Consider for example an equal-weighted portfolio of two independent assets with the same volatility: its DR is equal to $\sqrt{2}$, and to $\sqrt{N}$ for $N$ independent assets.

$DR(\bf{w})=\frac {\sum_i{\it{w}_i\sigma_i}} {\sigma(\bf{w})}$

Answer 2

The problem of the selecting the best portfolio (according to some risk measure) with a limited number of assets can be formulated as a mixed integer linear or quadratic program and is reviewed in the recent paper "Portfolio selection problems in practice: a comparison between linear and quadratic optimization models". It can be solved for reasonable sizes by several of the best optimizers like CPLEX or XPRESS. However, in the case of 5 stocks out of 10 there are only 252 possible possible different subsets (namely 10 choose 5) and they could be all exaustively explored with repect to the risk measure of preference by any personal computer.

Answer 3

If you only need to pick 5 out of 10 and want equal weights then just enumerate all 252 possibilities (as pointed out above) and compute the portfolio volatility

$(\textbf{1}'K^{(i)}\textbf{1})^{1/2} = \left( \sum_{ij}K^{(i)}_{ij} \right)^{1/2}$,

where $K^{(i)}$ is the covariance matrix for the $i$th subset. Then use whatever subset gives the lowest portfolio volatility. Here you are minimizing portfolio volatility so you will be biased towards lower volatility stocks. If you don't care about volatility per se and just want to minimize the contribution to portfolio risk related to correlation (somewhat loosely defined) then you can use the Most Diversified Portfolio (MDP) method. This method aims to minimize the diversification ratio

$\frac{w'\sigma^{(i)}}{\left(w'K^{(i)}w\right)^{1/2}} =\frac{\sum_j\sigma_j}{ \left( \sum_{ij}K^{(i)}_{ij} \right)^{1/2}}$

Again, just plug in values for each subset and use whatever gives the largest value.

Personally, I would argue that a few aspects of what your doing are inefficient.

1. Why equal weights? If you have a covariance matrix then you can almost always find less risky portfolios. Each stock has a different volatility so equal weights tends to take too much risk in more volatile stocks.
2. Why consider only your top 10? It is possible your best 5-stock portfolio includes stocks outside of your top-10 rankings due to correlations.
3. Instead, consider attempting to generate expected returns for your stocks. You can do this by running a simple linear regression using your sorting metric.

As has been pointed out, the full mean-variance optimization is hard to solve when you have a cardinality constraint and a large number of stocks to consider. A common approach is to employ $l_1$ norm based methods. The gist of it is instead of solving the standard mean-variance QP

$\min_w \{ \lambda w'Kw - r'w \}, w \geq 0, \sum_i w_i = 1$,

drop the budget constraint and add an $l_1$ penalty, i.e.

$\min_w \{ \lambda w'Kw - r'w + \gamma ||w||_1 \}, w \geq 0$.

As you slowly increase $\gamma$ the $w$ vector will get sparser. Stop once you only have 5 non-zero values. Afterwards, re-scale the weights to sum to one. This version of the problem is a convex relaxation of the actual cardinality constrained problem. The $l_1$-norm penalty can also be motivated as the solution to a robust portfolio optimization problem where returns are uncertain, but satisfy a box constraint.