better estimator of volatility for small samples

One commonly used sample estimator of volatility is the standard deviation of the log returns.

It is indeed a very good estimator (unbiased, ...) when the sample is large.

But I don't like it for small sample as it tends to overweight outliers in log returns.

Do you know if any other statistical dispersion measure that can be use to estimate the volatility of a stock? (I don't care about statistical properties; I just want it to estimate differently / better the daily risk of this stock.)

PS: I have already tried to use the norm 1 instead of the Euclidean norm. Any other idea / remark?

• en.wikipedia.org/wiki/Median_absolute_deviation seams interesting. Mar 31 '11 at 8:13
• anyone has used the concept of entropy for financial time series? Mar 31 '11 at 8:21
• "As we all know, the best known sample estimator of volatility is the standard deviation of the log returns.". Really? Evidence? Mar 31 '11 at 10:02
• Concerning entropy: This seems to be an interesting question in its own right - why don't you ask it as a separate question?! Mar 31 '11 at 10:57
• @Shane: Agreed! Apr 1 '11 at 1:51

You could use something like the interquartile or semi-interquartile range, which is somewhat more insensitive to extremes. This is a better measure to use if your data is skewed, however if your data is normally distributed it is still better to use the standard deviation.

http://en.wikipedia.org/wiki/Interquartile_range

• if which data is skewed? The sample of the entire population? Apr 1 '11 at 2:07
• I am referring to the sample data Apr 1 '11 at 2:29
• It is an interesting point, because a small sample is skewed by nature Apr 1 '11 at 2:53

You mention "daily" risk, so I'm assuming you're looking at a daily frequency. Yang-Zhang Volatility (Drift-independent Volatility Estimation Based on High, Low, Open and Close Prices) fits the bill for what you're asking, it takes into account intraday fluctuations as well.

• I guess it is anyway applicable for intraday or monthly data if you have a OHLC time serie. Apr 1 '11 at 1:53
• It will indeed dilute the effect of an outlier as we use more data than the close to close returns. But still we may have this "hat shape" of length the number of data in your rolling Yang-Zhang vol Apr 1 '11 at 1:57

In order to suppress the effect of outliers, you can use indicator transform or rank transform. Once you convert your data in those form then you can find the volatility. Personally, I like indicator transform as it is easy, more popular in many applications and gives good results.

• "more popular in many applications and gives good results." Do you have a reference for this? Apr 1 '11 at 1:58
• Where can I find the definition of a "sequential indicator transform"? Sorry for my lack of knowledge, it is the first time I hear about it. Apr 1 '11 at 2:56
• webs1.uidaho.edu/geoe428/files/GeostatSec7.pdf: You dont need to read all of it. Just get the idea of making a cut-off limit and using binary numbers 0 and 1 to populate your distribution. This way you can subdue the effect of outliers.
– user98
Apr 1 '11 at 4:20
• That paper has moved - would you be able to suggest another [link/paper]? Nov 20 '14 at 13:43

There are waaaaayyy better estimators than $$Var(log(Close_{t+1}/Close_{t}))$$. This close-close estimator is unbiased, and has a data efficiency defined as $$1$$.

The Parkinson estimator uses high and low prices only (useful when you don't trust your open and close prices, or don't have them). It has a data efficiency of about $$4$$. The expected variance from "true" is four times smaller than close-close.

The Yang-Zhang estimator use open, high, low, and close prices, and has a data efficiency of about $$14$$.

Many other estimators exist with different benefits and drawbacks. Nice math-based discussion of the different approaches in the Yang-Zhang paper.

However there is no magic bullet.

For small sample size, you are going to want to artificially inflate your sample size using machine learning hacks. You can turn it into a regression problem, where you think of things that generally should correlate with volatility (time of day, for instance), and use a regressor to estimate the estimate (meta-estimate?), which essentially means estimating the volatility using more samples than what you previously were limited to. However you will lose mathematical promises like unbiased-ness, and quality of estimate will need to be checked empirically.