How can one estimate the performance of a trading strategy on out of sample dataset? Yes, the good old model selection problem. Everyone knows sharpe ratio of your in-sample dataset by itself is a poor metric for the task at hand.

Could other share How they solve this? Perhaps a combination metric and process?

  • 2
    $\begingroup$ Combinatorially Purged Cross Validation... presented here (I hadn't noticed before but it seems to be another of our regular contributors!): quantoisseur.com/2019/11/05/… $\endgroup$
    – StackG
    Aug 16, 2020 at 10:54
  • $\begingroup$ I second this :) $\endgroup$ Aug 16, 2020 at 14:15
  • $\begingroup$ Hi @Quantoisseur. I don't quite get this. I understand the purpose of purging and embargo, but what is a path? What is this method trying to solve again? Also, perhaps I'm missing something here. In each train-test set, what are we even doing? And at the end of the cross-validation process, what did we found? Eg. best features? $\endgroup$ Aug 17, 2020 at 13:21
  • $\begingroup$ Paulsen & Sohl's Sharpe Ratio Information Criterion is applicable to portfolio problems, giving an unbiased estimate of the signal-noise ratio of the selected portfolio. A modification by Pav converts that into confidence intervals. More importantly, all statistics have standard error, you will likely have very small alpha, and shouldn't expect exotic cross-validation to be a magic bullet. $\endgroup$ Aug 18, 2020 at 22:55


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.