# How can I reproduce the experimental verification of the “False Strategy” theorem plot?

I recently came across the following blog post talking about the importance of back-testing overfitting, and a plot claiming to be an experimental verification of the False Strategy theorem.

The plot shown is: I would like to reproduce this plot to better understand the issue, but cannot find the original source, nor any info how to produce such a plot.

How can I reproduce this plot?

PS. This question may also be related to this one.

• There was a recent blog post on this topic at sharperat.io . The idea is to use the beta distribution to get a p-value, then plug that into the quantile function of the Sharpe ratio. – steveo'america Jun 19 '18 at 22:25
• So, in R , to get the median line for Sharpe measured on 1 year of daily data, do something like: SharpeR::qsr(qbeta(0.5, shape1=1:1e6, shape2=1), df=251, ope=252). Replace the 0.5 with the quantile of your choice. – steveo'america Jun 19 '18 at 22:27

Up to the presentation details, the combination of beta and Sharpe distribution function gives the plot data. Below is the code to compute and plot the median value and a ribbon between the 25th and 75th quantile, where the backtests are over a single year.

require(SharpeR)
require(dplyr)
require(ggplot2)

bt_len <- 252     # length of backtest
days_py  <- 252   # number of days per year
back_lens <- exp(seq(log(1),log(1e6),length.out=1000))

# compute 0.25, 0.5, and 0.75 quantiles
qv <- data_frame(nbacktest=unique(round(back_lens))) %>%
mutate(q25=SharpeR::qsr(qbeta(0.25,nbacktest,1),df=bt_len-1,ope=days_py),
q50=SharpeR::qsr(qbeta(0.50,nbacktest,1),df=bt_len-1,ope=days_py),
q75=SharpeR::qsr(qbeta(0.75,nbacktest,1),df=bt_len-1,ope=days_py))

ph <- qv %>%
ggplot(aes(x=nbacktest,y=q50,ymin=q25,ymax=q75)) +
geom_line() + geom_ribbon(alpha=0.25) +
scale_x_log10() +
labs(x='number of independent backtests',
y='maximal Sharpe, annualized',
title='maximal Sharpe over many independent 1 year backtests, median and IQR')
print(ph) 