How to properly cross-validate when optimizing SVM classification?

Question

I'm using SVM binary classification to predict movement of NASDAQ stock prices. My question is regarding cross-validation. I will divide the training data into V subsets. Training will be performed on (V-1) subsets and then prediction on the V-th subset is recorded. This will be done V times.

(1) Is the best measure of accuracy the average of all V outcomes? Or perhaps the minimum?

(2) Should subsets be equal length? Random?

(2a) Can subsets overlap?

(3) Do I need to ensure that each subset has at least some minimum percentage of each class outcome (1 or 0) for the results to be valid?

Answer 1

The cross-validation procedure does not turn on the choice of algorithm.

1. Yes - calculate the prediction error of the fitted models when predicting the V'th part of the data. Combine the V estimates of prediction average using a simple average.

2. Subsets should be randomly sampled (roughly equally sized). 2a. Subsets should not overlap.

3. No. As long as the sampling is random you are OK.