Using regression for binomial prediction of tomorrow's return

Question

I completed a challenge which asks the user to predict tomorrow's market return. The data available is prices data and the model must be logistic regression. They call it "machine learning" but it's pure regression. I derived a number of features from the data (e.g. momentum) etc.

The challenge implies that I need to train a model to predict returns out-of-sample. This is "extrapolation" which is not what regression is usually used for. Is this a valid use of regression?

I split my train/test data into 80%/20%. The split is performed by random sampling. What I mean is that for example, if I have data for 2010-2020, I don't use 2010-2018 for training and 2018-2020 for test but rather I randomly picked elements from the data such that 80% is train. This means that 2010-01-05 might belong to the train dataset but 2010-01-06 (next day) might belong to the test dataset.

My questions are:

1. Are my points about the use of regression here valid?
2. Does my test/train split mitigate the extrapolation problem with regression?
3. Does the random sampling of train/test cause problems due to the timeseries nature of the data?

Answer 1

(1) Regression is classic machine learning, and the general goal of ML is to train a model that can generalize to unseen (out of sample) data. Also, in the context of your challenge and in practice, people indeed care more about predicting the future (extrapolation) than fitting perfectly what has happened in the past, so using regression for this is valid.

(2, 3) Random split will not solve your problem, it will directly leak future information to your model and thus increase overfitting (returns can have high autocorrelation), due to the timeseries nature of the data.

A better cross validation setup is to use walk-forward data split. You could, for example, use TimeSeriesSplit or Purged K-FOLD cross validation with embargo to avoid leak between folds.

After you have the right validation setup, your main problem is to design features that can really explain future returns and are robust enough to go through market regime changes, or update your model regularly.