To estimate high frequency tick data stock intraday volatility, I have read Robert Almgren's notes7.pdf


where he talks about the bias free estimator by Zhou:

$Z = \sum ((y_j - y_{j-1})^2 + 2(y_j - y_{j-1})(y_{j+1} - y_j))$

where $y_j$ is the log return of the price at time $j$.

However, this expression sometimes yields negative volatility. We see that the first term is a square which is always positive, but the second term $2(y_j-y_{j-1})(y_{j+1}-y_j)$ can be negative. So how do I treat this estimator? I want a positive volatility, not a negative! I am thinking of just applying absolute value on the second term, but that does not sound right. Any suggestions?


In any finite sample, it is always possible for the Zhou estimator to return a negative number, even though we know the unobservable parameter being estimated is non-negative. This is a well known issue in the academic literature. There are several approaches to dealing with this problem:

1) Ignore it. (I don't like this one). It is particularly nefarious if you ever move to a multivariate setting and you suddenly end up with a covariance matrix that is not positive semi-definite. (try taking the inverse of it and watch everything go to hell in your code).

2) Set your final estimator equal to $max(v, 0)$, where $v$ is the Zhou estimator. In practice, there is nothing wrong with this ad hoc estimator, although a pure theorist might get cross about the fact that the asymptotic distribution is affected. From a coding perspective, you might even want to use $max(v, \alpha)$ for some small positive constant $\alpha$ so you don't accidentally end up dividing by zero anywhere. NOTE: please do not take the absolute value of the autocovariance portion of the estimator, as you mention in the question. This makes no sense from an estimation perspective and will result in a heavily biased estimator that is inconsistent even under ideal modelling assumptions.

3) Use a different intraday-data based estimator of volatility that doesn't suffer from this problem. I strongly recommend this option. The Zhou estimator was state-of-the-art in 1996 (and that paper itself was astonishingly pre-scient - it took another decade for everyone else to catch on to the problems Zhou tried to solve in that paper), but a lot of work since then has demonstrated that it will be heavily biased for many high-frequency datasets, see e.g. Hansen, Lunde (2006) "Realised Variance and Market Microstructure Noise". Probably the most popular estimator at the present point in time is the realised kernels estimator of Barndorff-Nielsen, Hansen, Lunde, and Shephard. The original 2008 paper is a bit heavy going for a non-theorist, but have a look at Barndorff-Nielsen, Hansen, Lunde, and Shephard (2009) "Realized Kernels in Practice" - it is much more friendly. (I have an implementation of this estimator in the Julia language based off this paper).

This estimator also has the nice property that if you use a Parzen kernel (see the above reference for more detail), then the estimator will always be non-negative. Be warned, for other types of kernel function, especially flat-top kernels, there is no guarantee of non-negativity. See eg footnote 2 of the above-mentioned paper.

If you want something simpler in that you can implement quickly, then just use a low sampling frequency realized variance, e.g. 5-minute realized variance or maybe 10-minute. Since this is just a sum of squared intrady returns, it is guaranteed to always be non-negative and will, in my experience, provide a better estimate than the Zhou estimator anyway.

  • $\begingroup$ Wow! :) At last a really helpful answer! Thanks so much for helping me! You mentioned you have a Julia implementation? Can you share it with us/me? I would really appreciate it, as the mathematical mumbo jumbo can be really daunting in some papers. O_o $\endgroup$ – Orvar Korvar Sep 20 '15 at 22:59
  • $\begingroup$ @OrvarKorvar It's a bit tricky at the moment. The routine depends on several modules that are currently proprietary. I've been meaning to split some of the popular high frequency estimator code (like realised kernels) out into its own module and post it to my public github page, but spare time is something I am really lacking at the moment. I've set a reminder to come back here and post a link to the github repo if I do manage to get it done. Sorry, best I can do at this point. My own fault for writing the code the way I did in the first place... $\endgroup$ – Colin T Bowers Sep 21 '15 at 3:25

I think it's ok: the total correction is $\approx -N\eta^2 \sim N^{-1/2},$ which tends to zero for large $N.$ So as long as it is a correction to the estimator $Q > 0$ you are ok.

  • $\begingroup$ If I use this on high frequency stock data, sometimes the Zhou estimator yields a negative realized variance which means the volatility is imaginary. I think this is a problem? $\endgroup$ – Orvar Korvar May 21 '15 at 18:42
  • $\begingroup$ if $Q > 0$ and $N$ is large, $Q + C/N^{1/2} > 0,$ right? $\endgroup$ – LazyCat May 21 '15 at 19:09
  • $\begingroup$ Hmm... what are you trying to say? That negative volatility is ok? Do you have any suggestions on how to handle the bias correcting term which can be negative? I am thinking of just using the absolute value, but that is not really good. $\endgroup$ – Orvar Korvar May 21 '15 at 21:31
  • $\begingroup$ I am saying, that the term you are worried about is not the whole volatility estimate, but rather a correction. And for large $N$ this term (even when negative) won't make your volatility estimate negative. $\endgroup$ – LazyCat May 21 '15 at 21:35
  • $\begingroup$ Intraday Realized Variance for a whole day of High Frequency tick data, sometimes is negative for certain days. The correction term is too negative, yielding negative variance for the whole day. Do you think I have a bug in my implementation, or can it occur that intraday realized variance becomes negative for a whole day? We see that the realized variance can be negative, it is not a square. $\endgroup$ – Orvar Korvar May 25 '15 at 22:56

Well for real hf stock tick data, i do get negativa volatility. So it can happens for large N

  • $\begingroup$ Hi user16313, what do you mean by negative volatility, as using the common it must be positive? $\endgroup$ – Bob Jansen May 22 '15 at 7:36

If you calculate this estimator point by point, it could very well go negative. But the essence of this estimator is that statistically, it hovers around the realized variance, so it "estimates" the realized variance well. Even though the realized variance is by definition non-negative, the estimated distribution does not have to be.

In practice can you really use this estimator in place of real-time variance? It's like shooting a target through a blurred lens. You could miss by a lot.

A potential use case of this estimator though, is that if you only care about high variance moments in your trading algorithm. Negative values in this case become irrelevant. Zhou's estimator is easily calculated online as market data streams in.

I found this presentation by Jim Gatheral in 2006 on this subject. His conclusion is that Zhou's estimator outshined other estimators in the survey.

By the way I learned something from other top answers, especially the realized kernel method. It looks to me that Zhou's original paper set up the ground, and others generalized the problem into a kernel problem. Now it just becomes a study of different kernels.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.