1
$\begingroup$

I recently came across the following blog post talking about the importance of back-testing overfitting, and a plot claiming to be an experimental verification of the False Strategy theorem.

The plot shown is:

enter image description here

I would like to reproduce this plot to better understand the issue, but cannot find the original source, nor any info how to produce such a plot.

How can I reproduce this plot?


PS. This question may also be related to this one.

$\endgroup$
  • 2
    $\begingroup$ There was a recent blog post on this topic at sharperat.io . The idea is to use the beta distribution to get a p-value, then plug that into the quantile function of the Sharpe ratio. $\endgroup$ – steveo'america Jun 19 '18 at 22:25
  • $\begingroup$ So, in R , to get the median line for Sharpe measured on 1 year of daily data, do something like: SharpeR::qsr(qbeta(0.5, shape1=1:1e6, shape2=1), df=251, ope=252). Replace the 0.5 with the quantile of your choice. $\endgroup$ – steveo'america Jun 19 '18 at 22:27
2
$\begingroup$

Up to the presentation details, the combination of beta and Sharpe distribution function gives the plot data. Below is the code to compute and plot the median value and a ribbon between the 25th and 75th quantile, where the backtests are over a single year.

require(SharpeR)
require(dplyr)
require(ggplot2)

bt_len <- 252     # length of backtest
days_py  <- 252   # number of days per year
back_lens <- exp(seq(log(1),log(1e6),length.out=1000))

# compute 0.25, 0.5, and 0.75 quantiles
qv <- data_frame(nbacktest=unique(round(back_lens))) %>%
    mutate(q25=SharpeR::qsr(qbeta(0.25,nbacktest,1),df=bt_len-1,ope=days_py),
                 q50=SharpeR::qsr(qbeta(0.50,nbacktest,1),df=bt_len-1,ope=days_py),
                 q75=SharpeR::qsr(qbeta(0.75,nbacktest,1),df=bt_len-1,ope=days_py)) 

ph <- qv %>%
    ggplot(aes(x=nbacktest,y=q50,ymin=q25,ymax=q75)) +
    geom_line() + geom_ribbon(alpha=0.25) + 
    scale_x_log10() +
    labs(x='number of independent backtests',
             y='maximal Sharpe, annualized',
             title='maximal Sharpe over many independent 1 year backtests, median and IQR')
print(ph)

ggplot of max SR vs # backtests.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.