How do you distinguish “significant” moves from noise?

Question

How do you distinguish between losses that are within the normal range for day-to-day shifts and situations with a real potential for loss? The specific application I have in mind is pattern recognition-based algorithmic trading.

Answer 1

Measuring expected shortfall (also known as conditional value-at-risk) answers the simpler question of "what is my average expected loss at the i-th quantile?" given the empirical distribution of returns. A variation is value-at-risk which measures the loss at the i-th quantile.

Arguably you could leave at this this and you have your answer.

You probably want a more robust estimate of your risk.

In this case you can use the bootstrap methodology. When you compute confidence intervals from a random sample, the statistics are themselves random variables. Indeed, your sample of returns itself is one of many possible samples. Each possible sample gives a possible value of Value-at-Risk, mean returns, etc. Although we observe one set of statistics using all your data it was selected at random from many values so it is therefore a random variable.

Enough with the theory - the procedure is not very difficult.

1. Define some statistic(s) of interest. Let's say it is value-at-risk.

2. Create a re-sample. You do this by sampling from your distribution of returns WITH replacement. Sample 'n' times where n is the number of observations. (You can actually sample 2n, 3n... if you'd like)

3. Calculate the statistic of interest on the re-sample.

4. Repeat steps #2 and #3 a couple thousand times.

5. Since the re-samples are independent of each other (b/c we re-sampled with replacement), the statistic you calculate in #4 is itself a random variable. You can now construct a confidence interval for the statistic of interest by measuring the standard error of the estimate and the t-statistic.

The bootstrap let's you answer your qualification "within a normal range". The bootstrap recognizes that the empirical distribution is itself a sample from an unknown population.

You can read more about the bootstrap here.

Answer 2

The most basic strategy is beta-based quantiles. That is to say, you first control for losses on your individual stock versus overall market performance. (Your trading strategy may or may not wish to hedge away the market factor using, say, SPX futures). Then you choose a quantile, call it the 5th percentile, beyond which you consider a move to be significant.

Symbolically, you are looking at an individual stock return $r_S$, a market return $r_M$, and a simple linear model that you have typically fitted historically

$ r_S = \beta r_M + \epsilon $

On any given day, you can now compute the idiosyncratic return $ r_i = r_S - \beta r_M $ which will have come from the same distribution as $ \epsilon $. You may or may not have turned your historically observed residuals $ \epsilon_i $ into a normal (or other continuous) distribution, but either empirically or normally it is now trivial to see if $ r_i $ is below the 5th percentile for $ \epsilon $.