2
$\begingroup$

I am setting up a backtesting using a walkforward optimization model to find out if a trading strategy performs well or not and I would like to clarify some doubts:

First of all what is the correct width of the window (training + testing period) to consider (is there any standard or empirical method to find it out) ?

What should the portion of the window to reserve for training and the one for testing (80%/20% , 50%/50%) ?

Is it correct that the window shift period should be equal to the length of the testing period?

|--------------------------------------------------|   <--- available time series data
|----------|-----|     <--- time window
      |----------|-----|
  ^        ^         ^      
  |        |         testing data
  |        training data
  window shift period

For each training data set the strategy will be optimized based on certain parameters. Then the optimized strategy will be used on the corresponding testing data set.

This approach could potentially produce different set of optimized parameters values for each time window.

So what is the best indicator or method to find out if a strategy performed well and what is the best way to choose the parameter values among the ones resulting from the various time window optimization ?

Thank you for your help

P.S. : any link/advice about good articles or books on this particular subject it is appreciated!

$\endgroup$
1
$\begingroup$

So for starters, there is not 'best time window' for backtesting. I would argue that it depends upon your data. If you are training an ML model that requires vast quantities of data to train on, then you should allocate a higher proportion of your time-series data to training the model. Whilst this may leave you with little test data during your backtest, you could always annualise your returns or observe how the wider market performed during the same period.

I would argue against having several training data sets. If you train your algo on several segments then it could lead to either- overfitting or survivorship bias. You want your algorithm to generalise over the entire period, instead of specialising in just one period.

I would insist on leaving the final segment worth n%, this becomes your backtest data and essentially your test set for all intents and purposes. The preceding set you can do as you wish for your training. An axiom in ML is the 60/40 split or 70/30 split for training/ testing. Again, it is up to as you need enough data to train on.

You could use several ML based metrics as evaluation factors, as your are fitting a regression you could use the RMSE for example. You could also use more finance based metrics to assess your algorithm empirically. For example- alpha, beta, maximum drawdown defficiency (MDD) or SR. I hope this answer helps.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I'm with you when you saying 'best time window' : that is a parameter that should be chosen in relation of the trading strategy you are testing. _If you train your algo on several segments then it could lead to either- overfitting _ but this is what the walk forward analysis is aimed to avoid: i mean the analysis of a large number of train/test periods should decrease the fact that the resulting performance is a product of chance. regarding evaluating factor, i pick up the strategy that maximize pnl $\endgroup$ – fabio May 4 at 16:11
  • $\begingroup$ Ah I see what you mean by walk forward now, that makes sense. But the last sentence you mention- pick the strategy that maximises pnl. This could very well be a working strategy, or it could be that you have cherrypicked the strategy that performed best during the backtest. It is crucial that you can distinguish between these two. $\endgroup$ – Hamish Gibson May 4 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.