8
$\begingroup$

I am reading Advances in Financial Machine Learning by Marcos López de Prado. In chapter 11 The Dangers of Backtesting, exercise 11.5 asks:

We download P/E ratios from Bloomberg, rank stocks every month, sell the top quartile, and buy the long quartile. Performance is amazing. What’s the sin?

Indeed, what is the problem? I don't see anything wrong with the method.

Improve this question
$\endgroup$
6
  • 10
    $\begingroup$ Beware of look ahead bias. The earnings of ACME Corp. for the second quarter of 2016 are not known on the last day of the second quarter of 2016. You need a database that has info about when the earnings were released and also when the earnings were amended/revised (if they were). Most db only have the latest info and the period to which it is related. $\endgroup$
    – nbbo2
    Jul 26, 2021 at 17:17
  • 8
    $\begingroup$ Also Bloomberg removes companies that have gone out of business. Another bias. You need a database that includes dead companies. In summmary Bloomberg is great for looking up the most recent info on current companies, it is not suited for historical backtests. $\endgroup$
    – nbbo2
    Jul 26, 2021 at 17:45
  • 4
    $\begingroup$ Small nitpick: it's hard to rank on P/E, since E can be zero or negative. Practically, it's easier to rank E/P. I think the sin is that you look at E now, ignoring future growth of E. Anyway, you may like people.stern.nyu.edu/adamodar/pdfiles/invfables/ch3new.pdf and also doi.org/10.2307/2326304 S. Basu. Investment Performance of Common Stocks in Relation to Their Price-Earnings Ratios: A Test of the Efficient Market Hypothesis (1977). $\endgroup$
    – Dimitri Vulis
    Jul 27, 2021 at 1:35
  • 3
    $\begingroup$ @noob2 Your comments are helpful. Please copy them into an answer. $\endgroup$
    – Flux
    Jul 27, 2021 at 5:24
  • 1
    $\begingroup$ What is "performance"? You should not only look at raw returns but also compute risk-adjusted returns and various risk measures $\endgroup$
    – Kevin
    Jul 27, 2021 at 6:49

1 Answer 1

Reset to default
6
$\begingroup$

Proper backtesting is difficult, because of various biases that easily slip into the results if you are not careful.

For example how do you compute historical P/E's. Well, you have historical E's and historical P's so you can divide P by E (as Dimitri Vulis suggests it is better to divide E by P, as most academic studies do). However beware of look ahead bias (using information in the backtest that was not available to investors at the time). The earnings of ACME Corp. for the year ending Q2 2016 are not known on the last day of the second quarter of 2016. They become public with a lag. You need a database that has info about when the earnings were released and also when the earnings were amended/revised (if they were). Most db only have the latest info and the period to which it is related. Instead you need "point in time" information, i.e. what information was available at various past points.

Also Bloomberg removes companies that have gone out of business. Another bias (survival bias, i.e. you are testing only the companies that survived). You need a database that includes dead companies.

In summmary Bloomberg is great for looking up the most recent info on current companies, it is not necessarily suited for historical backtests (at least without considerable additional work).

Improve this answer
$\endgroup$
1
  • 2
    $\begingroup$ On Bloomberg‘s defence you can get both point-in-time index members as well as fundamentals :-) $\endgroup$
    – oronimbus
    Aug 3, 2021 at 19:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.