# How is target return optimized, in practice?

Suppose we have $$A$$ a $$T \times N$$ matrix of daily returns for an asset universe of $$N$$ items, $$b$$ a $$(T,)$$ vector of daily returns for a target asset, $$x$$ a $$(N,)$$ weighting vector. We want a portfolio that replicates daily returns $$b$$ as closely as possible, subject to the constraint that portfolio weights $$x$$ sum to 1 and $$x$$ is non-negative (no shorting). One might use this for tax loss harvesting or hedging.

If we first consider the unconstrained case:

$$\min_x \sum_t^{T} (A \cdot x - b)$$

This is nothing more than least-squares approximation / regression, whose objective can be re-written as:

$$|Ax-b|^2 = (Ax-b)^T(Ax-b) = x^T{A^T}Ax - 2x^TA^Tb + b^2$$

Taking the derivative wrt $$x$$ and setting to 0,

$$2A^TAx - 2A^Tb = 0 \\ x = (A^TA)^{-1}A^Tb$$

To handle equality constraints such as $$x$$ summing to 1, and inequality constraints like $$x$$ being non-negative, one can set up a constrained least squares problem, solve the KKT matrix, and still derive a closed form solution, albeit one that has cubic complexity due to necessity of inverting matrices.

However, the portfolio manager may have a few other constraints - maybe they prefer to keep $$x$$ relatively sparse, so they can just buy about 3-4 assets rather than allocating a small percentage across thousands of assets. There are more complex techniques like ridge regression, lasso regression, and so on, but these start to involve stepwise optimization (e.g. coordinate descent).

My question is one about how the portfolio optimization industry actually optimizes in practice: is it more similar to analytic convex optimization with simple inequality / equality constraints, and maybe a bit of hacks on top of the Least-square regression (for instance, rounding off small values of $$x$$), or is more step-wise optimization the norm? Given a sufficiently complex objective, would it make sense just to switch to a more black-box optimization algorithm, like a genetic algorithm or some kind of stochastic gradient descent optimizer?

• I have seen the latter become normal at a number of places - e.g. pick as many constraints as you want and chuck your objective function into scipy.optimize, of course while monitoring convergence and checking whether you are hitting constraints etc. Jun 4, 2021 at 16:45

Your practical problem here is the insistence on non-negative weights, that sum to 1. There is no closed-form equation that solves this problem.

However, it can be done via iterative methods. You weight each stock i by Ai that is unbounded, Its portfolio weight Wi is e(AI)/sum(e(Ai)). The weights sum to 1 and are all positive.

SumW = sum(e(Ai))

dWi/dAi = Wi x (1-Wi) / SumW

dUtility/dAi thus becomes above x dUtility/dWi