9
$\begingroup$

To estimate high frequency tick data stock intraday volatility, I have read Robert Almgren's notes7.pdf

http://www.cims.nyu.edu/~almgren/timeseries/notes7.pdf

where he talks about the bias free estimator by Zhou:

$Z = \sum ((y_j - y_{j-1})^2 + 2(y_j - y_{j-1})(y_{j+1} - y_j))$

where $y_j$ is the log return of the price at time $j$.

However, this expression sometimes yields negative volatility. We see that the first term is a square which is always positive, but the second term $2(y_j-y_{j-1})(y_{j+1}-y_j)$ can be negative. So how do I treat this estimator? I want a positive volatility, not a negative! I am thinking of just applying absolute value on the second term, but that does not sound right. Any suggestions?

Improve this question
$\endgroup$
1
  • $\begingroup$ Do you have a functional link to that Robert Almgren note? Thanks. $\endgroup$
    – Hans
    Sep 15, 2020 at 18:15

4 Answers 4

Reset to default
7
$\begingroup$

In any finite sample, it is always possible for the Zhou estimator to return a negative number, even though we know the unobservable parameter being estimated is non-negative. This is a well known issue in the academic literature. There are several approaches to dealing with this problem:

1) Ignore it. (I don't like this one). It is particularly nefarious if you ever move to a multivariate setting and you suddenly end up with a covariance matrix that is not positive semi-definite. (try taking the inverse of it and watch everything go to hell in your code).

2) Set your final estimator equal to $max(v, 0)$, where $v$ is the Zhou estimator. In practice, there is nothing wrong with this ad hoc estimator, although a pure theorist might get cross about the fact that the asymptotic distribution is affected. From a coding perspective, you might even want to use $max(v, \alpha)$ for some small positive constant $\alpha$ so you don't accidentally end up dividing by zero anywhere. NOTE: please do not take the absolute value of the autocovariance portion of the estimator, as you mention in the question. This makes no sense from an estimation perspective and will result in a heavily biased estimator that is inconsistent even under ideal modelling assumptions.

3) Use a different intraday-data based estimator of volatility that doesn't suffer from this problem. I strongly recommend this option. The Zhou estimator was state-of-the-art in 1996 (and that paper itself was astonishingly pre-scient - it took another decade for everyone else to catch on to the problems Zhou tried to solve in that paper), but a lot of work since then has demonstrated that it will be heavily biased for many high-frequency datasets, see e.g. Hansen, Lunde (2006) "Realised Variance and Market Microstructure Noise". Probably the most popular estimator at the present point in time is the realised kernels estimator of Barndorff-Nielsen, Hansen, Lunde, and Shephard. The original 2008 paper is a bit heavy going for a non-theorist, but have a look at Barndorff-Nielsen, Hansen, Lunde, and Shephard (2009) "Realized Kernels in Practice" - it is much more friendly. (I have an implementation of this estimator in the Julia language based off this paper).

This estimator also has the nice property that if you use a Parzen kernel (see the above reference for more detail), then the estimator will always be non-negative. Be warned, for other types of kernel function, especially flat-top kernels, there is no guarantee of non-negativity. See eg footnote 2 of the above-mentioned paper.

If you want something simpler in that you can implement quickly, then just use a low sampling frequency realized variance, e.g. 5-minute realized variance or maybe 10-minute. Since this is just a sum of squared intrady returns, it is guaranteed to always be non-negative and will, in my experience, provide a better estimate than the Zhou estimator anyway.

Improve this answer
$\endgroup$
11
  • 1
    $\begingroup$ @Hans I haven't followed the academic literature particularly closely over the past 5 years. However, based on my own personal messing around with financial data I'd say that it is hard to beat 5-minute realized variance, and for more liquid assets you can bring that down to 1 or 2-minute realized variance. These estimators have the bonus of being very simple to implement. I think these estimators perform well because trying to model autocorrelations (like realized kernels etc do) introduces a lot of additional estimation error due to the increased implied parameter space. $\endgroup$
    – Colin T Bowers
    Sep 11, 2020 at 0:45
  • 1
    $\begingroup$ @Hans I'm also rather fond of a procedure I designed during my PhD, described here. However, it is hard to see this procedure ever becoming popular in the main-stream academic literature since it is hard (impossible?) to do the asymptotic theory in the popular continuous-time semi-martingale plus noise framework. You need to use discrete-time modelling to justify the estimator theoretically and for whatever reason the academic literature tends to avoid that for high frequency data (but that is another story...) $\endgroup$
    – Colin T Bowers
    Sep 11, 2020 at 0:47
  • 1
    $\begingroup$ Thank you for the informative update. What about even higher frequency data than 1 minutes tick data? Does the same methodology apply? I do not quite understand your statement "I think these estimators perform well because trying to model autocorrelations (like realized kernels etc do) introduces a lot of additional estimation error due to the increased implied parameter space." You seem to say the estimators perform well BECAUSE they introduce more error. It seems paradoxical. What am I missing? I will look into your PhD thesis. Thank you very much! $\endgroup$
    – Hans
    Sep 11, 2020 at 20:04
  • 1
    $\begingroup$ @Hans The higher the frequency of the financial returns, the more likely they will exhibit non-zero autocorrelation (typically negative autocorrelation due to microstructure effects like bid-ask bounce). Realized variance (of any frequency) assumes the autocorrelation is zero. Thus you should only use realized variance at frequencies where the zero autocorrelation is mostly satisfied (typically 5 mins, but maybe 1 or 2 mins for very liquid assets). Estimators like Realized kernels use every transaction. So by design they have to work with non-zero autocorrelation in the data (contd) $\endgroup$
    – Colin T Bowers
    Sep 13, 2020 at 12:01
  • 1
    $\begingroup$ @Hans All the ultra high frequency estimators (realized kernels, pre-average realized variance, mult-scale realized variance etc) have various estimation techniques for dealing with the autocorrelation. But you don't get something for nothing in statistics. By trying to estimate the autocorrelation (in some sense) you introduce estimation error. So there is a trade-off. Maybe the additional estimation error is small, so it is worth it to use all the data. But maybe the additional estimation error is large, in which case it is better to just use low freq realized variance - which I prefer. $\endgroup$
    – Colin T Bowers
    Sep 13, 2020 at 12:05
1
$\begingroup$

If you calculate this estimator point by point, it could very well go negative. But the essence of this estimator is that statistically, it hovers around the realized variance, so it "estimates" the realized variance well. Even though the realized variance is by definition non-negative, the estimated distribution does not have to be.

In practice can you really use this estimator in place of real-time variance? It's like shooting a target through a blurred lens. You could miss by a lot.

A potential use case of this estimator though, is that if you only care about high variance moments in your trading algorithm. Negative values in this case become irrelevant. Zhou's estimator is easily calculated online as market data streams in.

I found this presentation by Jim Gatheral in 2006 on this subject. His conclusion is that Zhou's estimator outshined other estimators in the survey.

By the way I learned something from other top answers, especially the realized kernel method. It looks to me that Zhou's original paper set up the ground, and others generalized the problem into a kernel problem. Now it just becomes a study of different kernels.

Improve this answer
$\endgroup$
0
$\begingroup$

I think it's ok: the total correction is $\approx -N\eta^2 \sim N^{-1/2},$ which tends to zero for large $N.$ So as long as it is a correction to the estimator $Q > 0$ you are ok.

Improve this answer
$\endgroup$
6
  • $\begingroup$ If I use this on high frequency stock data, sometimes the Zhou estimator yields a negative realized variance which means the volatility is imaginary. I think this is a problem? $\endgroup$
    – Orvar Korvar
    May 21, 2015 at 18:42
  • $\begingroup$ if $Q > 0$ and $N$ is large, $Q + C/N^{1/2} > 0,$ right? $\endgroup$
    – LazyCat
    May 21, 2015 at 19:09
  • $\begingroup$ Hmm... what are you trying to say? That negative volatility is ok? Do you have any suggestions on how to handle the bias correcting term which can be negative? I am thinking of just using the absolute value, but that is not really good. $\endgroup$
    – Orvar Korvar
    May 21, 2015 at 21:31
  • $\begingroup$ I am saying, that the term you are worried about is not the whole volatility estimate, but rather a correction. And for large $N$ this term (even when negative) won't make your volatility estimate negative. $\endgroup$
    – LazyCat
    May 21, 2015 at 21:35
  • $\begingroup$ Intraday Realized Variance for a whole day of High Frequency tick data, sometimes is negative for certain days. The correction term is too negative, yielding negative variance for the whole day. Do you think I have a bug in my implementation, or can it occur that intraday realized variance becomes negative for a whole day? We see that the realized variance can be negative, it is not a square. $\endgroup$
    – Orvar Korvar
    May 25, 2015 at 22:56
0
$\begingroup$

Well for real hf stock tick data, i do get negativa volatility. So it can happens for large N

Improve this answer
$\endgroup$
1
  • $\begingroup$ Hi user16313, what do you mean by negative volatility, as using the common it must be positive? $\endgroup$
    – Bob Jansen ♦
    May 22, 2015 at 7:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.