Goal: A team and I are looking to build a model that performs a predictive action for the state of the market on day
T + n, using the data at hand on day
T. To build this model, I'm using an EOD market data source going back until the early 1990s.
Moreover, we are looking to separate the dataset into training and testing subsets, in order to optimize a model parameter, then to eventually test its performance in the out-of-sample subset.
Problem: The initial attempt is to split the dataset by calendar year, and randomly assign each year into either the testing or training set. However, the following issues have been voiced:
- Since our model is attempting to predict
ndays into the future (we can assume
nis less than, say 15 or 20, but likely greater than 2-3), our training dataset needs to pull the market state from
ndays in the future in order to do our analysis. This would seem to indicate the we either need to pull
ndays from the testing dataset, or that we would need to drop the last
ndays from training set
- Any given
n-day span during the year may have some significance for our analysis, whether it's an earnings report, or otherwise. In particular, the
n-day window that ends the calendar year is considered an important period for our analysis, so dropping these
nborder points is not ideal (and also might result in systematic model bias)
Question: Is there a proper way to partition training and testing datasets, given that our analysis requires that we use a the datapoint that occurs
n days in the future?