The trading strategies that are going to backtest well are the ones that pick the winners from the past. For example, if a trading strategy simply bought apple stock it would backtest extremely well. The bias is easy to spot in this specific scenario, but for complex trading strategies there could be hidden bias of this type.
I am wondering if it is possible to eliminate this bias by backtesting in a different manner. What if you used the historical data to build a statistical model of the market or part of it. Such a model could include the risk free rate, historic volatilities, and correlations between stocks, etc. You could then run a monte Carlo simulation where you backtest against multiple, statistically generated historical data. You could use this to develop statistics about how the strategy works in a broader sense.
Any thoughts on this idea?
-- Reference : https://www.quantopian.com/posts/backtesting-thoughts