I'm attempting to optimize a reinforcement learning system to maximize risk adjusted returns. I have currently defined the reward as the differential Sharpe ratio at each step: the influence of the return at time t on the cumulative Sharpe ratio.

This was defined in this paper [Reinforcement Learning for Trading, by John Moody and Matthew Saffell, NIPS, 1999], however I am interested in maximizing Sortino ratio instead. The assets I am trading are highly volatile but also heavily skewed towards positive returns; the Sharpe ratio of the strategy is only 2.5 but the Sortino is ~11. So, maximizing for Sharpe here doesn't make much sense. The derivation of this differential Sharpe ratio is: enter image description here

I would like to adapt this only take into account the variance of my downside risk. Any help or further reading would be great.

  • $\begingroup$ It is not clear that maximizing Sharpe "doesn't make much sense". Upside variation can induce "lost opportunity cost" regret in an investor. $\endgroup$ – steveo'america Jan 22 '18 at 19:34
  • $\begingroup$ Have you been able to find a solution for this, I am facing the same problem. $\endgroup$ – Niklaus Mar 5 '18 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.