I have invested a few years in learning R and have developed a number of Monte Carlo backtesting scripts. My question is this: In general, for a person with some experience writing R code who is interested in Monte Carlo backtesting of various strategies, is it worth the time to go through quantstrat's steep learning curve, or would one be better off putting that time into continuing to develop and modify one's own backtesting scripts?

If this question is off topic, I apologize. I can't seem to find another place to ask it.

  • $\begingroup$ Can't answer your question, but: In case you haven't seen it, look at "The Probability of Backtest Overfitting" by Bailey et. al. $\endgroup$
    – jd8
    Aug 3 '17 at 5:57
  • $\begingroup$ I think that he knows about backtest overfitting, otherwise he wouldn't be using Monte Carlo simulation in his own scripts. $\endgroup$
    – james42
    Aug 3 '17 at 6:02
  • $\begingroup$ @james42 Why do you make that connection? Ernie, have you? $\endgroup$
    – jd8
    Aug 4 '17 at 10:58
  • $\begingroup$ Usually avoiding over fitting boils down to validate the results with some kind of randomisation algorithm. Even cross validation can be imagined as a Monte Carlo method :) $\endgroup$
    – james42
    Aug 4 '17 at 11:01
  • $\begingroup$ I don't know Quantstrat, but what I find objectionable about packaged solutions is that they force you to test the kinds of strategies that the authors have in mind, which 1000's of other people are also testing. I think you need to be creative in coming up with your strategy (holding different instruments for different lengths of time based on different data and indicators than everybody else is doing) if you want to be successful. Which will require writing your own code IMHO. $\endgroup$
    – Alex C
    Aug 5 '17 at 21:27

It never hurts to have additional tools at your disposal and I would therefore suggest learning quantstrat. Before you start hitting the books ask yourself:

  • Are you trying to enhance back testing by mitigating overfitting risk?
  • Are you trying to build more complexity into the strategy itself by layering more signals?
  • Do you want to generate extreme time series data replicating economic collapse and how your strategy would behave under those circumstances?

The question you're asking is ambiguous in that regard. Depending on what you are trying to do, your approach would vary (i.e. to use quantstrat or not).

What aspect of your backtests are your trying to improve?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.