1
$\begingroup$

Hi I am trying to use financial technical Indicators for forecasting, using machine learning models. The usual approach in time series cross validation is to use a moving window or growing window. The methodology I am using is described in the following steps

  1. Calculate technical indicators TA1, TA2, ....TAN for the whole historical data set, using lag 1
  2. Then use simple feature selection method like finding out the cross correlation between the independent variables, and remove variables with cross correlation above a certain threshold
  3. Lag the input variables by one
  4. Then train the the Machine learning model using a moving window, then reports its performance on the train set
  5. Test its performance on a test set which was not used in the training process

The issue I am facing is that the results are highly optimistic. My question is should I calculate the technical indicators separately for the train and test and then use them, or should I calculate them for the whole data set in the beginning and then divide them into train and test set.

$\endgroup$
  • $\begingroup$ I have also tried to do some of this myself but I kept running into overfitting problems. I would first optimize the train dataset & then use a walk forward approach but the problem with this type of optimization is overfitting. I think that is why you are getting very optimistic results. $\endgroup$ – Rime Dec 26 '14 at 1:07
1
$\begingroup$

In terms of forecasting, it is VERY difficult to forecast financial time series especially using ML models. One of the "successful" papers that I have seen use a classifier approach (e.g. forecasting extreme returns).

See: http://algorithmicfinance.org/2-1/pp45-58/

The above being said, your model structure would assume that the parameters are stable across time. Why not use a VARMA approach using the TA indicators see here: http://users.monash.edu.au/~gathana/slides/isf07.pdf

$\endgroup$
0
$\begingroup$

Careful when you use machine learning techniques in financial time series. You are implicitly assuming that the trend that you spot on the data you train your model on, the same trend will be there in future time series.

In addition to what KKB suggested, there is another model called "Echo State Network" (a family of RNNs) you should look at. But then again you are advised to train on brief data. Reason being - what happened 2 years back may not be relevant now, but what's happening in past few months are relevant as they reflect recent events. It's a maze really. You have to use your intuition.

Also use the confidence interval to judge how meaningful your estimated value is. A probability distribution is quite helpful in gauging the effectiveness of the prediction.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.