I'm working on a balanced, binary classification problem in a time-series (financial) dataset. I am using K-fold cross validation that is adapted for time-series (so that I'm never using future data to predict past data).
I have tried many algorithms, such as SVM, RandomForest and K-Nearest Neighbors. While all of them can achieve good results in cross validation, NONE of them have generalized well to the test set.
I use the cross validation to run grid-search feature selection and hyperparameter tuning simultaneously to find the best combination, but again - I have not achieved any generalization.
Do you have any ideas as to why this might be? Any general advice for dealing with this kind of scenario?