0
$\begingroup$

I have around 5 years of data of an index containing many features on a daily basis. I want to classify whether the index will move up or down the next trading day (up or down movement is determined by next day open/close price). I am using an SVM classifier for this classification. What could be some essential steps which need to be followed? I suppose since I am using financial data, there would be some deviation from the traditional method of applying machine learning. I have the following steps in mind:

Prepare data containing all features at-hand and direction variable.

Feature engineering: That is creating various features using transformations like log, % change on various available features. I have not incorporated this step but will do it later when I have a decent running model.

Feature Selection: How to go about this? I first am using various features which I think have predictive power but how to go about it systematically.

Walk forward Modelling: After selecting features I have the data which I will be using for my model. I am using the first 500 days of data to train the model and testing it on the next day and then using data from day 2 to day 501 to train the model and then testing it for day 502, and so on until I reach the present day. I make sure that I scale my training data in the range 0-1 and using the same scaler for test data before running the model. I use the default parameters of SVM in sklearn right now.

I then check the performance of my model by using various metrics obtained from confusion matrix-like accuracy, F1 score, etc.

For applying SVM I am using this reference which tells to use cross-validation to choose C and gamma. How can we use CV here if we have financial time series data? What else should I include in my steps?

$\endgroup$
1
  • 1
    $\begingroup$ you forget to list a very important step in ML: "build a baseline model". I suggest a logistic regression. It allows you to measure the gain of using a non linear method. $\endgroup$
    – lehalle
    Apr 12, 2022 at 3:11

1 Answer 1

2
$\begingroup$

For true cross-validation you need $N$ separate calibration sets, for example one data set per year. You can then test performance on, say, the month of data following each calibration set.

In feature engineering (FE), the essential problem is that if you systematize FE to "try many things", it becomes much harder to avoid the pernicious issues of overfit that occur in high-noise situations like financial return predictions. In my opinion you are better off avoiding engineering until after you have satisfied yourself that what you attempt is even possible.

Your conception of "walk-forward modeling" is reasonable and important, so kudos to you on that score. Note that training data scaling needs to be done in the same manner, lest you introduce one of the many types of "look-ahead bias".

Needless to say, many brilliant people have pursued SVMs for this kind of prediction before you, and have arbitraged away any profits they found. Unless your data set contains unusual or private categories of data, I recommend you view this project as an intellectual/academic exercise more than a profit opportunity.

$\endgroup$
1
  • 1
    $\begingroup$ Do you know of any study of this kind of applying SVM, which is available online? Would provide me with a perspective on a framework one might follow. $\endgroup$ Mar 18, 2021 at 10:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.