Q1. How to create an 'overlap' when we predict a stock price tomorrow based on information today?
- According to the book 'Advances in Financial Machine Learning' written by Marcos Lopez de Prado, the concept 'purging' is introduced to reduce the information leakage from the training dataset to train dataset.
One way to reduce leakage is to purge from the training set all observations whose labels overlapped in time with those labels included in the testing set. I call this process “purging.”
If my trading model is predicting stock price tomorrow using any information today, how can I perform "purging"?
To be more precise, I will give you an example. Let's say I use 3 companies' daily returns, GOOG, AAPL and MSFT to predict NASDAQ's daily return tomorrow. The input for my model is daily returns of 3 stocks, and the output is 1 daily return of NASDAQ tomorrow.
How can I create an 'overlap' between today and tomorrow?
Q2. Can 'train dataset' appear after the 'test dataset'?
In the figure 7.2 presented below, a train dataset is from the time period after the test dataset.
It seems bizarre to me because usually in finance, we predict future based on past information. We don't predict or forecast the past based on the future data.
As such, it is against my intuition to have a train dataset after the test dataset in terms of time period.