backtesting guide for research

Question

I am a master student in finance and I am working on my portfolio management thesis. Within my thesis I will have to backtest a portfolio strategy for a balanced portfolio.

I am looking for a guide/ paper that describes how to backtest an investment strategy. Preferably the guide/ paper is oriented towards research. I have never done any backtesting before and I am not yet aware about the pitfalls that could occur during the process. I would like to work with Matlab or Excel (Python would also be possible, but less preferred).

Answer 1

This was too long for a comment, so I'm writing it as an answer. I have provided some interesting literature that will give you insight into the common pitfalls of backtesting algorithmic trading strategies.

Marcos Lopéz de Prado on backtesting:

Marcos Lopéz de Prado provides some very good slides giving you a quick introduction to the goal of backtesting, before diving in to the common pitfalls of backtesting algorithmic investment strategies based on predictive models (this relates to portfolio backtesting as well). He argues that the hardest pitfall to avoid is the multiple testing problem (ie. adjusting your model/strategy based on multiple backtests is dangerous), and presents some solutions to avoid this problem. In general, his presentation is related to his own co-authored papers specified below:

• Bailey, David H., et al. (2014). "Pseudo-mathematics and financial charlatanism: The effects of backtest overfitting on out-of-sample performance".
• Bailey, David H., et al. (2016) "The probability of backtest overfitting".
• Bailey, David H., and Marcos Lopez De Prado (2014). "The deflated Sharpe ratio: correcting for selection bias, backtest overfitting, and non-normality".
• Bailey, David H., et al. (2015). "Statistical overfitting and backtest performance".

Alternative literature:

There's also Daniel P. Palomar's slides on backtesting that tells you seven sins of implementing quantitative investment strategies (survivorship bias, transaction costs, cost of shorting, multiple testing problem etc). He further gives an introduction to ways of doing a backtest, which includes cross-validation, walk-forward and k-fold cross-validation. In the slides, he also refers to some of the papers of de Prado, described above. The slides are from the course Portfolio optimization in R found here.

Alternatively, if you want a research-based paper you can take some inspiration from Victor deMiguel's paper Optimal Versus Naive Diversification: How Inefficient is the 1/N Portfolio Strategy?, detailing how mean-variance portfolio models fail to outperform the heuristic equal-weight portfolio out-of-sample. The study provides a way of comparing different portfolios and does not relate much to backtesting.

---

All in all, it will be a good idea to ask your supervisor for additional reading materials regardless of the above. He will point you in the right direction and might suggest well-known backtesting literature or methods he is familiar with.