time series - How to forecast expected volatility from high-frequency equity panel data?

I'm wading through the vast sea of literature on realized volatility estimation and expected volatility forecasting (see, e.g. Realized Volatility by Andersen and Benzoni, which cites 120 other papers, and Volatility by Bandi and Russell, which cites a slightly overlapping set of 120 papers).

I'm having a tough time finding research that specifically addresses the simultaneous estimation of a broad cross-section of equity volatility from high-frequency returns time-series. I'm looking for something along the lines of Vector Autoregression (VAR), but applying both sophisticated techniques developed for large equity panel estimation (thousands of volatilities and potentially millions of correlations being estimated) and using recent advances developed for efficient estimation using high frequency data.

What papers address the specific problem of forecasting the cross-section of equity volatility from high frequency data?

Answer

As far as I know the short answer is negative: there isn't a well developed theory of how to forecast cross-sectional realized volatility. From the perspective of statistics/econometrics, most of the recent research is still trying to find its way around estimation of cross-sectional realized volatility, and so far even in these area the progress is slow.

Bringing modern techniques to panel data amounts to being able to:

1. extract information irregularly spaced transaction data (UHFT or tick level),


2. deal with "microstructure noise",

in a multivariate setting where the additional problems of non-synchronous trading and high-dimensionality complicates the analysis, together with the usual hassle that comes with HAC estimators (such as the dimensionality issues that @QuantGuy mentions).

There are two main tools for tackling estimation: (1) the pre/post averaging approach of Ait-Sahalia, Mykland, Renault (and others) and (2) the kernel smoothing of Barndorff-Nielsen, Hansen (and others) [the third child, i.e. VAR and its crew, seems on the sideline of late, but I'd be happy to be proven wrong here]. Of these two approaches, only the second has matured a technology (Multivariate realised kernels) that is published (here).