How to calculate historical intraday volatility?

Sorry for what must be a beginner question, but when I went to write code I realized I didn't understand exactly how historical volatility, or statistical volatility, is defined. Wikipedia tells me "volatility is the standard deviation of the instrument's logarithmic returns", and logarithmic return is defined as $\ln\left(\frac{V_f}{V_i}\right)$, where $V_f$ is closing price and $V_i$ is opening price.

If I want to calculate the volatility of a minute bar, from the raw ticks, do I just use the first and last tick in that minute? If I use first and last ticks in the minute (i.e. bar open/close), I will have a single logarithmic return, so s.d. of that one value will be 0. In an answer to this question, the intraday volatility chart is described as U-shaped. Exactly what sums do I need to do to generate that intraday volatility chart from the day's ticks?

In R terms, is logarithmic return:

#x is xts object containing ticks
r = na.omit( lag(x)/x )
lnr = log(r)


Background: I have a stream of ticks, and as I turn them into minute (and higher period) bars (using R's xts module) I also calculate the mean and s.d. Is the standard financial measure of volatility different from standard deviation? If not, can one be derived from the other?

If the above definition of volatility is correct, my answer (based on eyeballing the plots, and on running cor) seems to be that they are really quite different; I'm still chewing over how that gels with the answers here.

-
I've cleaned this up a bit by editing your comments into your question and posting my comments as an answer. – Tal Fishman Dec 16 '11 at 3:01
@TalFishman Thanks for the edits Tal; I also moved my comment over to your answer. – Darren Cook Dec 18 '11 at 2:16
I am also very intersted in the Intraday vola calculator for gamma scalping purposes, can you please share it? Thanks Martin – Martin Mar 16 at 19:37

The expression you have is fine. But more generally, for the intraday volatility, I don't think there "the correct definition". More like, whatever works in the given context. I found the following notes by Almgren pretty useful:

http://cims.nyu.edu/~almgren/timeseries/notes7.pdf

-
Thanks @LazyCat Knowing that all the sources I've seen are being deliberately vague on implementation details, not just being brief, was the thing I needed to know. My eyes glazed over on parts of that PDF, but the comment at the top of p.10 (people generally use "ad hoc" sampling) and the suggestion at the end to compute across a range of parameters and not trust a result that varies a lot, was very helpful. – Darren Cook Dec 16 '11 at 1:19

I use Yhang Zhang measure for intraday volatility for timeseries with a rolling 5 or 10 day window. I wrote a C++ and vba implementation which I'm happy to share if you wish. Takes olhc data and gives an 'estimate' of the volatility. For intraday trading (gamma hedging), I found it is a fairly good estimator of the days range. But I would caution on whether it's a predictor of vol. Day to day intraday vol correlation tends to be small in my opinion.

I've added the code on request. It's probably not 100% robust, and there's probably some conditions on which it produces odd numbers or plain crashes. No implied guarantees

/**
Calculate the yzvol from the intraday data using a given window size
Note that this window will make the final array (size - window) in length
so the first 'window' dates in the dt_ array are to be skipped
@open - open prices
@high - high prices
@low - low prices
@close - close prices
@window - window size to do a rolling apply over
*/
std::vector<float> calculate_yzvol(
const std::vector<float>& open,
const std::vector<float>& high,
const std::vector<float>& low,
const std::vector<float>& close,
const size_t window) {
std::vector<float> yzvol;

const size_t element_num = open.size();
if (element_num - 1 < window) {
BOOST_LOG_TRIVIAL(warning) << "Number of daily prices " << element_num << " from DB is smaller than window size"  << window;
return yzvol;
}

std::vector<float> log_ho(element_num);
std::vector<float> log_lo(element_num);
std::vector<float> log_co(element_num);
std::vector<float> log_oc_sq(element_num);
std::vector<float> log_cc_sq(element_num - 1);
std::vector<float> rs(element_num);

std::transform(open.begin(), open.end(), high.begin(),std::back_inserter<std::vector<float>>(log_ho),[](auto open, auto high) {
return ::log(high / dailyopen);
});
for (size_t i = 0; i < element_num; i++) {
const float dailyopen = static_cast<float>(1.0 / open[i]);
log_ho[i] = log(high[i] * dailyopen);
log_lo[i] = log(low[i] * dailyopen);
log_co[i] = log(close[i] * dailyopen);
float oc = log(open[i] / close[i]);
log_oc_sq[i] = boost::math::pow<2>(oc);
rs[i] = log_ho[i] * (log_ho[i] - log_co[i]) + log_lo[i] * (log_lo[i] - log_co[i]);
}

for (size_t i = 1; i < element_num; i++) {
const float cc = log(close[i] / close[i - 1]);
log_cc_sq[i - 1] = boost::math::pow<2>(cc);
}

// Vol sum function
auto vol_sum = [](auto begin, auto end, auto window) {
const float window_factor = static_cast<float>(1.0 / (window - 1.0));
float sum = 0.0f;
for (auto i = begin; i != end; i++) {
sum += *i;
}
return sum * window_factor;
};
typedef std::vector<float> vf;
const vf close_vol = rolling_window<vf, vf>(log_cc_sq, window, vol_sum);
const vf open_vol = rolling_window<vf, vf>(log_oc_sq, window, vol_sum);
const vf window_rs_vol = rolling_window<vf, vf>(rs, window, vol_sum);

// Note that this window will make the final array (size - window) in length
const float result = ::sqrt(open_vol[i.index() + 1] + 0.16433 * close_vol[i.index()] + 0.835667 * window_rs_vol[i.index() + 1]);
yzvol.emplace_back(result);
});
return yzvol;


}

-
Ronnie - I would love to try your code. Appreciate if you can share it with me. – fooledbypattern Mar 8 at 19:50
Thanks. There have been a couple of requests for the implementation; are you able to add a link to a github page/blog post/whatever that shows it? Or, if it is compact enough, add it to your answer? – Darren Cook Mar 16 at 20:48
@fooledbypattern - here's the code in C++ attached. DarrenCook, hope this helps – Ronnie Mar 17 at 14:04

The equivalence you are trying to find can only exist in the framework of static volatility.

I think the problem is that in the real world, statistical volatility varies a lot with time; and worse off the relative rate at which it varies increases with smaller time increments.

So not only does the answer not apply in real-world markets, an estimation of its fiat theoretical existence becomes exponentially less precise with smaller time increments.

(I think the Heston model supports this because the random component of change in variance of stock price is proportional to the square root of its current self, although its not immediately obvious this follows.)

-

The main issue measuring intraday volatility is called "signature plot": when you zoom in, the volatility measure (i.e. empirical quadratic variations) explode.

Similarly you have the "Epps effect" for correlations: when you zoom in, the correlations collapse (it is at least a mechanical effect).

For the volatility a lot of models can correct this: - first a multiscale filter (use wavelets for instance) - then an additive noise model (the ZAM estimator, see Almgren's note) - or a random time observation (i.e. liquidity/tick based one) like this: http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1989281 - an Hawks model is also a nice solution if you like point processes, see for instance: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.187.1605

But the most important is what do you want to do with your volatility model?, which market dynamics do you want to capture? If it is market risk, it will not be the same if it is to compute the probability to cross a price barrier, or to compute a price interval for the price during the closing fixing.

-

On a theoretical level and for low frequency data (e.g. daily), your formula seems right. However, since you are talking about one minute bars, things may get a little messy. There is a vast literature on this, and empirically, things are complicated due to market micro-structure noise. Namely, you need to do consider jumps, errors, periods of low volume, high volume periods (e.g. market opening of US markets when you look at European markets), and many other things. A nice overview on the topic with some references to literature is here: http://economics.ouls.ox.ac.uk/13045/1/2008OMI11.pdf

-

Your code for volatility seems correct, if you want minute volatility, but is that really what you want? See this recent question on annualizing volatility from intraday data. Also, using first and last tick is what is generally done, but over very short time intervals such as a minute, you will have microstructure issues. Another question here deals with the state of the art in volatility estimation and forecasting. It doesn't really make sense to take SD of price. Vol is always something along the lines of SD of returns.

Regarding your comment that there is only a single open/close observation for a given interval (e.g., bar), you must plot an entire days worth of minute open/close squared returns as a function of time to see the U-shape. The volatility is the mean of squared returns. Historical volatility is defined by two parameters, the interval over which you take returns and the lookback period over which you average those squared returns. In your case, you may also sum rather than average all the squared returns for one day to obtain the "daily volatility measured over minute intervals."

Note I have assumed that returns have a mean of zero in the above. This is a very reasonable assumption for returns over such a short horizon.

-
Thanks for "Historical volatility is defined by two parameters"; now it becomes clear. When someone asks me for a volatility measure I now know I need to ask them two questions, not just one. – Darren Cook Dec 18 '11 at 2:16

Statistical volatility is the standard deviation of a window of log returns. For example, 30-day statistical volatility is the standard deviation of 30, one-day log returns. The log return comes from the assumption that log stock returns are normally distributed.

Statistical volatility differs from implied volatility which is the volatility input to some options pricing model (read: Black-Scholes) which sets the model price equal to the market, or observed price.

Statistical and implied volatility are used for different purposes.

Variance of course is the standard deviation of a random variable squared. Variance is has useful properties in the normal distribution (e.g. variance is additive while standard deviation is not) which is used extensively in modeling the dynamics of equity prices.

In measuring central tendencies of returns in the financial context, variance doesn't really make sense because variance is not in standardized units like standard deviation. Unless you're dealing with variance swaps or stochastic volatility models, you'll probably be dealing exclusively in standard deviation.

So to answer your question in short, calculate lag-number of log returns, take the standard deviation and that's the lag-period statistical volatility of your returns.

With all that said, this is a fairly basic question for this group and I'm not sure it will last...

-
Thanks for the answer. Cutting to your short answer, does "calculate lag-number of log returns" mean the same as the R code I've written? Or do I have it wrong? – Darren Cook Dec 14 '11 at 8:20
More questions, sorry. It is statistical, not implied, volatility I'm interested in. You are contrasting it with variance, not s.d.; does that mean the only difference between stastical volatility and variance is the square. I.e. stastical volatility and standard deviation are the same? Because the standard deviation of log returns in a time period and the standard deviation of actual prices in the same period appear to be quite different (i.e. peaks on the two charts are in different places). – Darren Cook Dec 14 '11 at 8:26
@DarrenCook it doesn't really make sense to take SD of price. Vol is always something along the lines of SD of returns. – Tal Fishman Dec 14 '11 at 12:18
@DarrenCook The code appears correct but I don't know R. Tal is right, we don't take the SD of raw prices because it doesn't make sense in a financial context. Yes, statistical volatility and the standard deviation of log returns are the same. And yes, variance is the square of SD. You have to think outside of strict statistics here and think within the context of financial markets and data. – strimp099 Dec 14 '11 at 14:41