Using Technical Indicators for forecasting Financial time series using Machine learning models

Question

Hi I am trying to use financial technical Indicators for forecasting, using machine learning models. The usual approach in time series cross validation is to use a moving window or growing window. The methodology I am using is described in the following steps

1. Calculate technical indicators TA1, TA2, ....TAN for the whole historical data set, using lag 1
2. Then use simple feature selection method like finding out the cross correlation between the independent variables, and remove variables with cross correlation above a certain threshold
3. Lag the input variables by one
4. Then train the the Machine learning model using a moving window, then reports its performance on the train set
5. Test its performance on a test set which was not used in the training process

The issue I am facing is that the results are highly optimistic. My question is should I calculate the technical indicators separately for the train and test and then use them, or should I calculate them for the whole data set in the beginning and then divide them into train and test set.

Answer 1

In terms of forecasting, it is VERY difficult to forecast financial time series especially using ML models. One of the "successful" papers that I have seen use a classifier approach (e.g. forecasting extreme returns).

See: http://algorithmicfinance.org/2-1/pp45-58/

The above being said, your model structure would assume that the parameters are stable across time. Why not use a VARMA approach using the TA indicators see here: http://users.monash.edu.au/~gathana/slides/isf07.pdf

Answer 2

Careful when you use machine learning techniques in financial time series. You are implicitly assuming that the trend that you spot on the data you train your model on, the same trend will be there in future time series.

In addition to what KKB suggested, there is another model called "Echo State Network" (a family of RNNs) you should look at. But then again you are advised to train on brief data. Reason being - what happened 2 years back may not be relevant now, but what's happening in past few months are relevant as they reflect recent events. It's a maze really. You have to use your intuition.

Also use the confidence interval to judge how meaningful your estimated value is. A probability distribution is quite helpful in gauging the effectiveness of the prediction.