• Given approximately 11 years of historical time-series data, to determine how much of this data should be reserved for in-sample and out-of-sample testing when using the data in the backtesting of a forecasting system
  • To become familiar with potential techniques, if any, that can be used to determine an in-sample and out-of-sample split to reduce the likelihood of the over-fitting modelling error


As observable from the chart seen below, the time-series data spans from March 2012 to October 2023. During this approximate 11.5 year period, various market states/ conditions can be seen. For example:

  • 2012 to 2014: up-trending market
  • 2014 to 2016: down-trending market
  • 2016 to 2018: sideways-moving market
  • 2019 onward: substantial up-trend and highly volatile, then steep down-trend

Additionally, from the end of 2019, the data was likely sensitive to real-world inputs including the effects of the COVID-19 pandemic which were not present earlier on in the sample.

enter image description here

Potential Approaches:

  • A simple 50:50 in-sample to out-of-sample split - the first half for testing and optimising, the second half for evaluating performance
  • A 70:30 split so the system is built around the majority of the data before moving on to performance evaluation (this could be more susceptible to over-fitting)
  • Given the various market conditions listed above, using a 50:50 split on each period i.e. the bullish market during 2012-2014 would be divided so the first half was for in-sample and the second half for out-of-sample. This process would be repeated for the bearish market from 2014-2016, the sideways market for 2016-2018, and the highly volatile and bullish market 2019 onward


Q1. Based on the characteristics of the time-series data, are there any ways that this data could be divided or would a 50:50 split be sufficient?

Q2. Are there any techniques that would help in determining an appropriate sample split, or is this very much a discretionary process that is unique to each sample set?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.