I'm looking for some direction on testing whether a simple entry signal has statistical significance.

Let's say this is my simple entry signal:

Buy when some indicator has a positive slope and is above zero on 3 time frames (5M,15M,60M)

Sell when the indicator has a negative slope and is below zero on 3 time frames.

How do I go about statistically testing this signal to see if there's any alpha to be extracted?

I was thinking something along these lines:

  1. plot the P&L distribution for the signal should it exit 1,2,3, N bars after the entry.
  2. plot the P&L distribution if the entry point had been random and the exit had been again 1,2,3, N bars after the entry.

I'm not really sure where to go from there.

How do I go about creating a "random" entry signal? Perhaps taking the time difference between the first and last quote and creating a pseudo random time to enter. The number of entries would have to be equal to the number of non-random signals created from the indicator. Also, I say pseudo random because it probably wouldn't be desirable if all these "random" signals happened to cluster in one short time interval.

Then, what characteristics of these distributions should I be looking for to establish if the indicator signal has any statistical significance in terms of potential alpha that can be extracted.

Any direction or reference would be appreciated. Being that there are so many different types of statistical tests that can be done, it's hard to find anything particularly relevant.

  • $\begingroup$ They way you ask your question makes me think that you are looking for somekind of bootstrap test. A related paper on that would be : Pairs Trading: Performance of a Relative-Value Arbitrage Rule Evan Gatev. Of course this will not actually tell you if you have found alpha. For this I think you should be looking at parametrical test. $\endgroup$ – Zarbouzou Oct 2 '12 at 9:31
  • $\begingroup$ Thanks for your reply. Looking into parametrical tests for trading signals I found "Trading with a 't' Signal Extraction Using Non-Parametric Newey-West Style t-statistics" (Cameron Rookley). It seems this may be a good example of what I'm looking for but I'll have to brush up on some statistics concepts before I can understand this paper fully. If anyone can propose a simple, practical parametric (or otherwise) test in the meanwhile that I can use to evaluate statistical significance of my particular signal, I would be much obliged. $\endgroup$ – Vazgen Oct 2 '12 at 21:32

I think the simplest way to achieve what you're looking for is through regression coefficient hypothesis testing.

  1. Perform linear regression on returns (y-axis) vs. dates (x-axis) over the desired time frames (do it once for 5 months, once for dataset w/15 months worth of data, and once for 60 months worth of data).

  2. As a result of regression, you will get coefficients for $y = mx + b$. To check if the slope is significantly different from 0.0, you would perform a t-test on m: $$ t = \frac{m - 0.0}{SE(m)} = \frac{m}{SE(m)},$$ where $SE(m)$ is standard error of $m$. Depends on how you're doing the regression, but it may be reported by the software.

  3. Using $n-2$ degrees of freedom (where $n$ is the number of points included in regression, look up $t$ values for the desired confidence level (typ., $\alpha=0.05$ for 95% confidence interval, CI). Let this value be $t_{critical}$.

  4. $m$ is significantly different from 0.0 if $t > t_{critical}$.

Slope's sign can be determined by adjusting the null hypothesis ($m > 0$, $m < 0$) & picking different $t_{critical}$ values.

...unless I misunderstood your question, that's one of the ways I see achieving what you're after.

  • $\begingroup$ Thank you for taking the time to help. It seems the test you proposed could be what I need. If I understand correctly, m is the regression coefficient and if it varies significantly from 0.0 then there is a correlation between the trade returns and time. But how does that relationship imply my signal is statistically significant? Perhaps I'm misunderstanding the interpretation of step 4. $\endgroup$ – Vazgen Oct 4 '12 at 5:15
  • $\begingroup$ I ultimately want to be able regress across a number of user defined metrics that are recorded right before the signal (bar range right before the signal, distance to previous high, value of a different indicator right before signal, etc) to see if there are relationships there that can yield new information about entries with a statistically high chance of turning profitable. $\endgroup$ – Vazgen Oct 4 '12 at 5:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.