For a university project, I have been working on a rather complex automated trading strategy, that levers machine learning techniques. I backtested the algorithm, as a result I have daily return data for various time windows (max. two years, it is a relatively new financial asset). I also simulated the performance of the algorithm on three months of test data.

Now I am writing the final report, but my daily supervisor wants more rigour i.e. statistical tests. E.g. he wants me to prove that the daily return on average is greater than zero or even the S&P500 daily return.

I believe this to be impossible. Let us assume all assumptions for whatever test are satisfied. Proving your average daily return to be higher than zero with an $\alpha = 5\%$, would imply it is statistically impossible to ever have a PERIOD (see EDIT below) with a negative return (which is definitely not the case for my strategy). Or thus, I believe my supervisor is asking me to prove that I have found the holy grail of automated trading strategies.

Two questions:

  • Do you agree with my above reasoning: statistically proving an average daily return higher than zero is nonsense?
  • Do you have any pointers or ideas about how I could somewhat prove or quantify that my algorithm works well, other than simulating over specific time periods and benchmarking my returns and Sharpe ratios against other assets and strategies.

EDIT: I meant period instead of day EDIT in response to @Olaf his comment: when I say statistically impossible I mean very improbable. Let us assume I would be able to reject ($\alpha=5\%$) the null hypothesis that the average daily return (e.g. calculated over 90 days) is smaller than zero. In case I would invest an equal amount every day, then given the above it is almost guaranteed that I will end up with more after 90 days. I tend to believe this is a very strong guarantee for an automated trading system.

  • $\begingroup$ "Proving your average daily return to be higher than zero with an α=5%α=5%, would imply it is statistically impossible to ever have a day with a negative return" What makes you say that? Statistical significance of a mean and variations around the mean are not the same thing. $\endgroup$ – Olaf May 5 '16 at 8:27
  • $\begingroup$ I'm voting to close this question as off-topic because this seems very confused regarding basic concepts, i.e. 'Basic financial questions'. $\endgroup$ – Bob Jansen May 5 '16 at 12:47

To have an average that is statistically > 0 does NOT imply that you can NEVER have a negative daily return: it means that, on AVERAGE, you have a positive return! You can have a lot of negative days, but if the positive ones more than compensate for the losses, you end up with an AVERAGE positive return (that can or cannot be statistically significant from zero: your test will answer this question).

Check if your model outperforms S&P500 in terms of cumulative returns over the period of interest, for example. Compare the expected returns and also the volatility (this is not a negligible variable) of your model w.r.t. those of S&P. I also think that simulation methods are useful tools to test a strategy.


Not the answer you're looking for? Browse other questions tagged or ask your own question.