1
$\begingroup$

I completed a challenge which asks the user to predict tomorrow's market return. The data available is prices data and the model must be logistic regression. They call it "machine learning" but it's pure regression. I derived a number of features from the data (e.g. momentum) etc.

The challenge implies that I need to train a model to predict returns out-of-sample. This is "extrapolation" which is not what regression is usually used for. Is this a valid use of regression?

I split my train/test data into 80%/20%. The split is performed by random sampling. What I mean is that for example, if I have data for 2010-2020, I don't use 2010-2018 for training and 2018-2020 for test but rather I randomly picked elements from the data such that 80% is train. This means that 2010-01-05 might belong to the train dataset but 2010-01-06 (next day) might belong to the test dataset.

My questions are:

  1. Are my points about the use of regression here valid?
  2. Does my test/train split mitigate the extrapolation problem with regression?
  3. Does the random sampling of train/test cause problems due to the timeseries nature of the data?
$\endgroup$
1
  • $\begingroup$ What is binomial about this problem? $\endgroup$
    – Dave
    Jul 22, 2022 at 4:07

1 Answer 1

2
$\begingroup$

(1) Regression is classic machine learning, and the general goal of ML is to train a model that can generalize to unseen (out of sample) data. Also, in the context of your challenge and in practice, people indeed care more about predicting the future (extrapolation) than fitting perfectly what has happened in the past, so using regression for this is valid.

(2, 3) Random split will not solve your problem, it will directly leak future information to your model and thus increase overfitting (returns can have high autocorrelation), due to the timeseries nature of the data.

A better cross validation setup is to use walk-forward data split. You could, for example, use TimeSeriesSplit or Purged K-FOLD cross validation with embargo to avoid leak between folds.

After you have the right validation setup, your main problem is to design features that can really explain future returns and are robust enough to go through market regime changes, or update your model regularly.

$\endgroup$
3
  • $\begingroup$ Hi: I may not totally follow what you're doing but the answer definitely depends on the horizon that you're trying to predict and the horizon that you're using to predict. For example, you can't just randomly sample days say if you're using a week of previous data to predict the next day's return. $\endgroup$
    – mark leeds
    Jul 22, 2022 at 7:41
  • $\begingroup$ Thanks for the answer. Would you not say that using regression on time series data is not the way to go? From what I remember, using regression on data that has autocorrelation is erroneous? I mean autoregression is a regression but the analysis of goodness of fit and the assumptions are different so it is dangerous to do standard regression with autocorrelated variables. $\endgroup$
    – s5s
    Jul 22, 2022 at 8:45
  • $\begingroup$ I think you were asking autoencoder but I'll comment anyway. The data is time series data only if you are modelling it as time series data. Say you have "features" ( X's ) and you are using those X's to predict say -1 for down, +1 for up and 0 for flat, then multinomial logistic regression on prices could be okay. It's when you're actually using a ts model that the notions of stationarity and correlation become issues. Since I don't know the details, I can't say if what you're doing is statistically problematic. Not that I could necessarily say anyway but more details are definitely needed. $\endgroup$
    – mark leeds
    Jul 24, 2022 at 15:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.