Short version
If you're trying to produce an asset allocation system, it feels pretty natural to split it into an estimation component that forecasts asset means and covariance, and a weighting component that uses those estimates to optimize some metric like maximizing the Sharpe ratio or minimizing the volatility. But the improving the estimation model and its estimates on their own terms (e.g. in a log probability sense) does not necessarily improve the the weighting strategy based on those estimates and sometimes actually degrades overall performance. What's the right way to think about and address this problem?
Longer version with concrete example
Suppose we have a 200-month span of returns data for two assets and we'd like to allocate investments across those assets in a way that minimizes volatility. For most of that span, returns are normally distributed with standard deviation X, but for a 30 month span in the middle, they're normally distributed with a standard deviation of 5X. Like the following:
If we fit a simple, static maximum likelihood model to this data, we can produce estimates of the mean and covariance. That mean and covariance are such that a minimum volatility strategy would have allocated roughly equally to both assets:
But obviously this simple, static model is imperfect. What if we introduce exogenous data and "improve" our model such that it can fit the period of elevated volatility for asset 0 but, crucially, does not improve our volatility estimates for asset 1? Then a minimum volatility strategy that knew these model-based estimates in advance might look like:
But this strategy is higher volatility than the constant, equal weights strategy from the simpler model! Even though the log probability of the dynamic model is higher.
There's clearly a disconnect here where improving the model in a log probability sense does not necessarily improve an asset allocation strategy (according to that strategy's own metrics) based on that model's estimates. In this particular case, improving the accuracy of the model with respect to one asset in absolute sense means that the model makes worse estimates about the relatively volatility of the two assets. And it seems like this sort of problem is inevitable given this structure because any given weighting strategy will be sensitive to different kinds of estimation error and no single estimation loss function can be optimal for every weighting strategy.
Is my understanding of this problem correct? Is there some standard way of thinking about it, a name for it, or any references to read more about it? Are there any tricks or reframings that penalize an absolute improvement in estimation accuracy for one asset which degrades relative accuracy across assets?
The only full solution I can think of is to integrate the estimation component and weighting component such that the estimator is fully informed by the weighter's loss function. But this seems undesirable in several ways:
- reduced sharing across different weighting strategies,
- interpretability,
- seems like it gives a much weaker training signal—one number like realized volatility or Sharpe ratio over an entire backtest span instead of return per asset per period in the backtest,
- harder to put in a Bayesian context
Is integrating the estimator and weighter in fact the "right" solution? Are the drawbacks listed accurate?
