I'm testing (out-of-time) my machine learning (ML) based strategy against a strong benchmark. As a performance metric, I'm using a custom rolling metric $M(t)$ which takes into account the portfolio value at time $t$ as well as other variables (inventory, time, etc). Visually, it is clear that the ML strategy is beating the benchmark since the performance difference is increasing in time quite steadily. However, I would like to back my findings up with an appropriate statistical test. I know nothing about the distribution of this metric, but the $M(t)$ increments (i.e. a differenced time series) corresponding to my ML strategy seem to be correlated with the $M(t)$ increments corresponding to the benchmark. In essence, I would like to prove that my strategy is consistently beating the benchmark, so I would like to find an appropriate statistical test. Perhaps something as simple as a t-test would do it?

  • $\begingroup$ White's Reality Check or Hansen's test for Superior Predictive Ability do well for this kind of problem of an arbitrarily defined metric against a benchmark. $\endgroup$ Mar 17, 2021 at 22:02


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.