Why do institutions backtest?

Question

I see that institutions still use backtesting by computing P&Ls over historical data and then compute some aggregating ratios to see whether a trading strategy is good or not even though it is not a rigourous approach at all. I mean, how can a trading strategy that happened to perform well in one sample path be guaranteed to perform as well out of sample ?

Answer 1

How can a trading strategy that happened to perform well in one sample path be guaranteed to perform as well out of sample?

I think you are having it backwards - this is how I do it:

1. Intuition about some economic, psychological, behavioral, technical etc. phenomenon.
2. Trying to make my intuition precise in the form of a hypothesis.
3. Trying to translate my hypothesis into a trading strategy.
4. Backtesting the trading strategy.
   • In case it does not work: Falsification of my intuition
   • in case it does work: More tests (robustness), and with some refined intuition back to step 1.

Basically this is how the scientific method works when doing research on the stock market. At least this is how it should be, so I somewhat agree with your insinuation that just data mining stock market data to find something is bad science ("Torture the data until they confess" ;-)

So, yes, it is not perfect - but it is the best we have to try to find the signal in the noise (and there is a lot of noise...)

A good starting point to understand more about this approach is this book:
Evidence-based technical analysis by David Aronson

It explains the whole process (including the complete statistical background).

See for a short summary of important points here: CXO Advisory

See for a comprehensive review here: Automated trading system

Answer 2

Mostly because of convention and tradition. As Student T mentioned earlier, part of this is that it is common practice. You report to your clients or managers how well something performed in the past; you cannot report to them how well it performed in the future. You may have thought of some useful forward-looking measures, but unfortunately the adoption rate in finance for these things is extremely slow. We still teach CAPM as the forefront in the leading business schools, even though this was introduced in the 1960s. We still credit novelists like Taleb for "discovering" non-Gaussian and black swan behavior in 2000s, even though the authors of the models that he critiques had themselves introduced jump diffusion models in the 1970s.

That said, I think you are making the implicit assumption here that regime shifts prevent you from applying your models out-of-sample:

How can a trading strategy that happened to perform well in one sample path be guaranteed to perform as well out of sample ?

There are persistent phenomena across all time horizons: The low volatility anomaly, the strong dominance of certain factors in explaining returns, volatility clustering, sector correlations, arbitrage between specific symbols, order book properties. It's the presence of these persistent behaviors that motivate people to attempt to extract insight from past data.