2
$\begingroup$

I want to perform features selection on my dataset. I've split my data into train, test and out-of-sample set. The dataset is time-series based, so the split is sequenced in the order that train set will be taken from the earliest segment of the dataset and the out-of-sample set will be taken from the latest segment of the dataset.

I apply spearman ranking on all 3 sets to obtain corrcoef as well as filter for only features with pvalue < 0.02. I noticed that all 3 sets reported a very different sets of correlated features. And in some features, corrcoef maybe positive in the train set, but on the test set is negative.

Unsurprisingly any model trained using the train set and validated using the test set, most likely will perform miserably on the out-of-sample. I suppose this is a very common problem in a quant finance dataset. How can I solve this?

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ You can't. Model selection (aka feature selection) is not a solved problem. Search on the topic over at stats.SE. Add in markets tending toward efficiency and it is even harder in finance than in plain stats. $\endgroup$
    – kurtosis
    Sep 15, 2020 at 6:09
  • $\begingroup$ @kurtosis if it is not solved how can others make it work? Surely, there are others that are doing it right or not? Or perhaps, the right path is to look for a set of features that stay consistent throughout the entire dataset. Then again, we can't tell what will happens outside our dataset. $\endgroup$
    – Kok Wooi Hew
    Sep 15, 2020 at 8:50
  • $\begingroup$ You try lots of things; you use all the experience and knowledge of theory you have. Ideally, a modeled grounded in the economics will change less over time. Hence why good modelers get paid so well. $\endgroup$
    – kurtosis
    Sep 15, 2020 at 18:05

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.