# How to forecast expected volatility from high-frequency equity panel data?

I'm wading through the vast sea of literature on realized volatility estimation and expected volatility forecasting (see, e.g. Realized Volatility by Andersen and Benzoni, which cites 120 other papers, and Volatility by Bandi and Russell, which cites a slightly overlapping set of 120 papers).

I'm having a tough time finding research that specifically addresses the simultaneous estimation of a broad cross-section of equity volatility from high-frequency returns time-series. I'm looking for something along the lines of Vector Autoregression (VAR), but applying both sophisticated techniques developed for large equity panel estimation (thousands of volatilities and potentially millions of correlations being estimated) and using recent advances developed for efficient estimation using high frequency data.

What papers address the specific problem of forecasting the cross-section of equity volatility from high frequency data?

• Isn't one of the more standard approaches to estimate a variance-covariance matrix? To get past the dimensionality issue you first build a factor model and create factor based covariance matrix. Wouldn't this result in a simultaneous estimation of the cross-section of equity volatility? This is covered in Grinold and Kahn Active Portfolio Management. You could innovate in how you estimate your betas (using a dynamic linear model for example). In this case your question becomes "How do I estimate Betas via Vector Autoregression"? – Ram Ahluwalia Oct 14 '11 at 17:22
• @QuantGuy these covariance matrices are typically estimated from daily (or lower frequency) data. When using higher-frequency data, handling asynchronous trading and lead-lag becomes crucial. The problem is that most volatility estimators using HF data focus on an individual instrument. – Tal Fishman Oct 17 '11 at 15:45
• If you have panel data then your data is not high frequency. – user2763361 Feb 22 '14 at 8:47

As far as I know the short answer is negative: there isn't a well developed theory of how to forecast cross-sectional realized volatility. From the perspective of statistics/econometrics, most of the recent research is still trying to find its way around estimation of cross-sectional realized volatility, and so far even in these area the progress is slow.

Bringing modern techniques to panel data amounts to being able to:

1. extract information irregularly spaced transaction data (UHFT or tick level),
2. deal with "microstructure noise",

in a multivariate setting where the additional problems of non-synchronous trading and high-dimensionality complicates the analysis, together with the usual hassle that comes with HAC estimators (such as the dimensionality issues that @QuantGuy mentions).

There are two main tools for tackling estimation: (1) the pre/post averaging approach of Ait-Sahalia, Mykland, Renault (and others) and (2) the kernel smoothing of Barndorff-Nielsen, Hansen (and others) [the third child, i.e. VAR and its crew, seems on the sideline of late, but I'd be happy to be proven wrong here]. Of these two approaches, only the second has matured a technology (Multivariate realised kernels) that is published (here).

• where could i read about microstructure noise? – Permian Feb 22 at 23:25

Abstract:

We introduce a blocking and regularization approach to estimate high-dimensional covariances using high-frequency data. Assets are first grouped according to liquidity. Using the multivariate realized kernel estimator of Barndorff-Nielsen et al. (2010), the covariance matrix is estimated block-wise and then regularized. The performance of the resulting blocking and regularization (‘RnB’) estimator is analyzed in an extensive simulation study mimicking the liquidity and market microstructure features of the S&P 1500 universe. The RnB estimator yields efficiency gains for varying liquidity settings, noise-to-signal ratios and dimensions. An empirical application of estimating daily covariances of the S&P 500 index confirms the simulation results.

• Thanks for the reference, it looks highly relevant, I'll check it out. You are literally seconds too late for the bounty, though, sorry. – Tal Fishman Oct 17 '11 at 15:51