Tell me more ×
Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It's 100% free, no registration required.
up vote 4 down vote favorite
1

I posted this on Cross Validated a month ago, but I didn't get any answers. Sorry for the second post!

I'm wondering about the whole process of testing/data-mining for a strategy and THEN testing on bootstrapped data. Does it make sense to bootstrap your data first and then data-mine for the best result?

This came to me because my current strategy had significant decay when I tested it against the bootstrapped data. The initial strategy is one that re-balances between the SPY and the TLT on a monthly basis. Back-tested on 6 years worth of data (or 72 total trades), the strategy has a Sharpe of 2.06. But when I test it against the bootstrapped data, the Sharpe ratio drops significantly to 1.04.

It seems like I'm ultimately looking to raise the 1.04 number, and to do that I'll need to

  1. start the data-mining process over again to find a new back-tested strategy

  2. test again against the bootstrapped data.

So can I skip step #1? Obviously this will require some significant computational power.

share|improve this question

1 Answer

up vote 2 down vote

It's good in the sense that the bootstrap will give you a more sobering view of reality. However, it is fairly difficult to perfectly capture all the relationships of financial assets via bootstrapping (dynamic auto and cross correlations, etc..). It would be great to see some comments on what people have been using. From a data mining perspective, you might want to look into cross validation techniques as it helps you to avoid over-fitting.

share|improve this answer
1  
Great point re: preserving the auto and cross correlations. Block bootsrap method discussed by Stambaugh appears effective in that regard but it seems to me that the research on the best bootstrapping method is not definitive – Quant Guy Nov 29 '12 at 2:52
Thanks for the comments. I had previously posted about preserving the correlations [link] stats.stackexchange.com/questions/40987/…, and ultimately I used the meboot package in R as it seemed to be the best approach. These results were similar to the results I had from the block bootstrap method. Regarding the cross validation - is there any way I can do this now that I've already mined for the strategy over years 2004-2012? – Julia Flores Nov 29 '12 at 3:05
I have heard good things about meboot, and started to run it, but I'm not so sure it captures both auto and cross correlations well. It doesn't really make any sense to do cross validation after the data set has been mined. The ideal method would be to train/validate/test over each slice fold set... so that every slice is not biased in any way by the slice that is left out. If you have a strategy, however, you could run the CV over parameters of the same strategy to evaluate robustness. – pat Nov 29 '12 at 3:25

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.