Backtesting with a walkforward approach

Question

I am setting up a backtesting using a walkforward optimization model to find out if a trading strategy performs well or not and I would like to clarify some doubts:

First of all what is the correct width of the window (training + testing period) to consider (is there any standard or empirical method to find it out) ?

What should the portion of the window to reserve for training and the one for testing (80%/20% , 50%/50%) ?

Is it correct that the window shift period should be equal to the length of the testing period?

|--------------------------------------------------|   <--- available time series data
|----------|-----|     <--- time window
      |----------|-----|
  ^        ^         ^      
  |        |         testing data
  |        training data
  window shift period

For each training data set the strategy will be optimized based on certain parameters. Then the optimized strategy will be used on the corresponding testing data set.

This approach could potentially produce different set of optimized parameters values for each time window.

So what is the best indicator or method to find out if a strategy performed well and what is the best way to choose the parameter values among the ones resulting from the various time window optimization ?

Thank you for your help

P.S. : any link/advice about good articles or books on this particular subject it is appreciated!

Answer 1

So for starters, there is not 'best time window' for backtesting. I would argue that it depends upon your data. If you are training an ML model that requires vast quantities of data to train on, then you should allocate a higher proportion of your time-series data to training the model. Whilst this may leave you with little test data during your backtest, you could always annualise your returns or observe how the wider market performed during the same period.

I would argue against having several training data sets. If you train your algo on several segments then it could lead to either- overfitting or survivorship bias. You want your algorithm to generalise over the entire period, instead of specialising in just one period.

I would insist on leaving the final segment worth n%, this becomes your backtest data and essentially your test set for all intents and purposes. The preceding set you can do as you wish for your training. An axiom in ML is the 60/40 split or 70/30 split for training/ testing. Again, it is up to as you need enough data to train on.

You could use several ML based metrics as evaluation factors, as your are fitting a regression you could use the RMSE for example. You could also use more finance based metrics to assess your algorithm empirically. For example- alpha, beta, maximum drawdown defficiency (MDD) or SR. I hope this answer helps.