# Turnover as a soft constraint for portfolio optimization

I am using cvxpy to do a simple portfolio optimization.

I implemented the following dummy code

from cvxpy import *
import numpy as np

np.random.seed(1)
n = 10

Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)

orig_weight = [0.15,0.25,0.15,0.05,0.20,0,0.1,0,0.1,0]
w = Variable(n)

mu = np.abs(np.random.randn(n, 1))
ret = mu.T*w

lambda_ = Parameter(sign='positive')
lambda_ = 5

constraints = [sum_entries(w) == 1, w >= 0, sum_entries(abs(w-orig_weight)) <= 0.750]

prob = Problem(Maximize(ret - lambda_ * risk), constraints)

prob.solve()

print 'Solver Status : ',prob.status

print('Weights opt :', w.value)


I am constraining on being fully invested, long only and to have a turnover of <= 75%. However I would like to use turnover as a "soft" constraint in the sense that the solver will use as little as possible but as much as necessary, currently the solver will almost fully max out turnover.

I basically want something like this which is convex and doesn't violate the DCP rules

sum_entries(abs(w-orig_weight)) >= 0.05


I would assume this should set a minimum threshold (5% here) and then use as much turnover until it finds a feasible solution.

Anyone any suggestions how this could be solved or re-written in a convex way?

EDIT: Intermediate solution

from cvxpy import *
import numpy as np

np.random.seed(1)
n = 10

Sigma = np.random.randn(n, n)
Sigma = Sigma.T.dot(Sigma)
w = Variable(n)

mu = np.abs(np.random.randn(n, 1))
ret = mu.T*w

orig_weight = [0.15,0.2,0.2,0.2,0.2,0.05,0.0,0.0,0.0,0.0]

min_weight = [0.35,0.0,0.0,0.0,0.0,0,0.0,0,0.0,0.0]
max_weight = [0.35,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0,1.0]

lambda_ret = Parameter(sign='positive')
lambda_ret = 5

lambda_risk = Parameter(sign='positive')
lambda_risk = 1

penalty = Parameter(sign='positive')
penalty = 100

penalized = True

if penalized == True:
print '-------------- RELAXED ------------------'
constraints = [sum_entries(w) == 1, w >= 0, w >= min_weight, w <= max_weight]
prob = Problem(Maximize(lambda_ * ret - lambda_ * risk - penalty * max_entries(sum_entries(abs(w-orig_weight)))-0.01), constraints)
else:
print '--------------   HARD  ------------------'
constraints = [sum_entries(w) == 1, w >= 0, w >= min_weight, w <= max_weight, sum_entries(abs(w-orig_weight)) <= 0.40]
prob = Problem(Maximize(lambda_ret * ret - lambda_risk * risk ),constraints)

prob.solve()

print 'Solver Status : ',prob.status
print('Weights opt :', w.value)

all_in = []
for i in range(n):
all_in.append(np.abs(w.value[i][0] - orig_weight[i]))

print 'Turnover : ', sum(all_in)


The above code will force a specific increase in weight for item[0], here +20%, in order to maintain the sum() =1 constraint that has to be offset by a -20% decrease, therefore I know it will need a minimum of 40% turnover to do that, if one runs the code with penalized = False the <= 0.4 have to be hardcoded, anything smaller than that will fail. The penalized = True case will find the minimum required turnover of 40% and solve the optimization. What I haven't figured out yet is how I can set a minimum threshold in the relaxed case, i.e. do at least 45% (or more if required).

I found some explanation around the problem in Multi-Period Trading via Convex Optimization by Boyd et al., in chapter 4.6 page 37.

Boyd Paper

• Is the sum of your original weights mean to be 0.25 or are we just meant to take that as an illustration? If so, it's going to be hard to build a portfolio that satisfies sum(w) == 1 with a turnover of <0.75. CVXPY is going to have to add at least 0.75 to the final portfolio weights. – Tim Wilding Mar 27 '18 at 10:21
• Adjusted the initial weights vector, that was me not paying enough attention. – ThatQuantDude Mar 27 '18 at 13:28

I’m going to focus on the maths here rather than the Python because that’s what I am most familiar with! The turnover constraint is often used as way to handle transaction costs when people are not certain of the exact value of the transaction costs. That gives an insight into how the turnover may be modelled as a transaction cost instead. You can then set up a transaction cost penalty function that should be able to satisfy your criteria of using as much turnover as possible without maxing out the turnover.

You can model the transaction cost explicitly by separating out a stock’s position into three components $w_0$, $w^+$, and $w^-$ - the initial weight, the purchase weight and the sales weight. The final position weight $w = w_0 + w^+ + w^-$. This is the simplest variant of a piecewise linear model of transaction costs (see Large Scale Portfolio Optimization with Piecewise Linear Transaction Costs by Potaptchik, Tuncel, Wolkowicz link for a more complicated example).

Using this formulation means that you have to rewrite your risk function, constraints and bounds. For example, you must set individual position bounds for the buy and sell components: $0 < = w^+ <= u – w_0$, and $-w_0 <= w^- <=0$. You would also have to add constraints to ensure that $w = w_0 + w^+ + w^-$ for each position.

If you use the piecewise linear approach, a turnover constraint could then be implemented as $\Sigma_i w_i^+ - \Sigma_i w_i^- <= t$. However, sticking with the transaction costs approach, you could provide each position with a cost, $c$, and you would replace your “ret” with $\Sigma_i w_{0,i}\mu_i + \Sigma_i w_i^+ (\mu_i -c_i) + \Sigma_i w_i^-(\mu_i+c_i)$. This is the same as your return function, but includes a transaction cost penalty function $\Sigma_i w_i^+c_i - \Sigma_i w_i^- c_i$. That function penalises larger transactions. Your only problem then would be to decide what value of $c$ to use in your model. Picking the right $c$ should ensure that the turnover constraint is minimised, but this can be quite complicated in practice and trial-and-error might be your best approach.

Using $c$ gives you a linear penalty function, but you could also extend this method to more complicated functions of $w^+$ and $w^-$ to restrict turnover. For example, you could use a quadratic function of the positions. That would reflect both uncertainty in the cost of trading and the fact that it might cost more to trade larger position sizes.

• Hi Tim thanks for your reply, I am actually using actual transaction costs already as part of my objective function (which I haven't included here), explicit + implicit. I want to be able to control transaction costs separately, currently I keep them below some bound but if I am running backtest (e.g. 100 rebalances) it fails because of infeasibility (at some stage). I want to use as little turnover to get a feasible solution, maybe it is easier to think about it in terms of a relaxation parameter here??? – ThatQuantDude Mar 28 '18 at 16:32
• Yes, you could relax the turnover constraint until you hit a minimum. That might require you to run multiple optimisations. Alternatively, rather than use a straight linear penalty function, you could use a quadratic penalty function for the turnover =$\lambda t^2$ ($t = \Sigma_i w_i^+ -\Sigma_i w_i^-$, i.e. all costs are equal in the above formulation) – Tim Wilding Mar 29 '18 at 13:49
• I added a half way solution, the code will determine the minimum required turnover for a feasible solution, however I still want to be able to set a lower threshold on that, i.e. do at least x% (or more if required) – ThatQuantDude Apr 3 '18 at 14:14