portfolio optimization with weights constraint in python

Question

I'm trying to optimize a portfolio using cvxpy. My original construction is the following:

w = Variable(n)
ret = mu.T * w
risk = quad_form(w, Sigma)
prob = Problem(Maximize(ret), [risk <= .01])

which is just maximize return under some risk constraint. However, I would like to also have a weights/leverage constraint, like the following:

prob = Problem(Maximize(ret), [risk <= .01, sum(abs(w)) <= 1.0])

However, when I add this constraint in many of my weights go to zero and the optimal portfolio is just concentrated in 2-3 assets. This is different from the case without this constraint which results in a much more diversified portfolio. I'm a little confused as to why the weights constraint causes this. Does anyone have any insight?

Answer 1

This is a bit more complex than adding additional constraints. This is a well known problem in markowitz optimization - if you don't treat your covariance matrix and expected return vector with great care, markowitz will often spray your weights against the edges and result in a very non-diversified portfolio.

I suggest robustly landscaping the literature - here is a good place to start:

http://www.ledoit.net/honey.pdf

"Estimating the covariance matrix of stock returns has always been one of the stickiest points. The standard statistical method is to gather a history of past stock returns and compute their sample covariance matrix. Unfortunately this creates problems that are well documented (Jobson and Korkie, 1980). To put it as simply as possible, when the number of stocks under consideration is large, especially relative to the number of historical return observations available (which is the usual case), the sample covariance matrix is estimated with a lot of error. It implies that the most extreme coefficients in the matrix thus estimated tend to take on extreme values not because this is “the truth”, but because they contain an extreme amount of error. Invariably the mean-variance optimization software will latch onto them and place its biggest bets on those coefficients which are the most extremely unreliable"

Answer 2

I had the same problems on matlab. I Guess that you need To put some boundaries as constraints:

cons=({'type':'eq', 'fun': lambda x:sum(x)-1})
Bounds= [(0.1 , 0.5) for i in range(0,nb_assets)]
Optim= scipy.optimize.minimize(fonction,
       InitialSolution,method='SLSQP',bounds=Bounds,constraints=cons)

This way you tell the optimization tool to find a more diversified solution.