Let's say I have a fairly volatile time series $X_t$ - it doesn't have any reason to show an upward / downward trend, but it does show drops and spikes from time to time. It can also change level (e.g. for a few weeks it will be at one level, but it will quickly drop down to a new level a few % points down and stay there).

I'd like to have a statistic (basically a time series Y_t) that signals a change in $X_t$ quickly. It should also converge to the same (or close to the same) level as $X_t$ with as little delay as possible. Finally, $Y_t$ should have as little volatility as possible (I'm aware there's a tradeoff between low volatility and responsiveness).

I've tried the rolling mean - if there's a short duration (say 1 period) drop in $X_t$, then the rolling mean reacts quickly to the drop, but shows a delay in reacting to the recovery in $X_t$ (i.e. when $X_t$ is restored to normal values). The delay is the same as the window size minus the drop duration.

At this stage, I'm considering two statistics. The exponential moving average with a window of $5$:

$$E_t=\frac{\sum_{i=0}^t(1-\alpha)^iX_{t-i}}{\sum_{i=0}^t(1-\alpha)^i}$$ where $$\alpha=\frac{2}{\text{window}+1}$$

The other statistic I'm considering is a combination of EMA and median. Let $M_t$ denote the rolling median with a window of $9$. Then $$C_t=aE_t+(1-a)M_t$$

To test their response to a drop, I did a quick simulation as follows:

a = pd.Series(np.hstack((np.ones(10)*0.81, np.ones(4)*0.77,
              np.ones(4)*0.72, np.ones(5)*0.77, np.ones(10)*0.81)))
x = 0.5
b = a.ewm(span=5).mean()
c = x*a.ewm(span=4).mean() + (1-x)*a.rolling(7).median()

figure(num=None, figsize=(27, 9)) 
plt.plot(a, marker="o")
plt.plot(b, marker=".")
plt.plot(c, marker=".")

enter image description here

By the time $C_t$ converges to the same values as $X_t$, the difference between $C_t$ and $E_t$ is really small (less than a percentage point).

$C_t$ has the feature that it won't drop as sharply as $E_t$ in response to a drop in $X_t$, but after a few periods it will abruptly drop down to the level of $X_t$. Similarly when $X_t$ recovers, $C_t$ initially shows a slower rise than $E_t$, but then suddenly shoots up, overtaking $E_t$ and catching up with $X_t$.

Now intuitively it seems that overall the EMA $E_t$ would be a better statistic, but I'm not sure. Is there any concrete, rigorous justification for the superiority of any one statistic over the other? For example, is it possible to show that maybe $C_t$ reacts similarly to two different scenarios, and is therefore unsuitable to differentiate between those scenarios?

Finally, given the requirements of quick response, quick convergence and low volatility, is there any other statistic I may have missed that's better than these two?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.