What are the most popular ways to compare volatility models?
Suppose I wanted to compare the forecasting accuracy of a GARCH(1,1) model with the historic 30 day volatility. What method should I use?
There is no one right answer to this question, but a common starting place is to compare the bias and variance of the forecast vs. the realized variance.
Take your forecasted variance $\hat y$ and regress them against the realized variance:
$y = \beta_0 + \beta_1 \hat y + \epsilon$
A few things that you want to see:
Make sure that you evaluate your model on the time horizon that is relevant to you. For instance, if you're rebalancing monthly, then it won't be especially important to predict the next day's volatility as much as if would if you were rebalancing daily.
As with any forecast, make sure that you have a sufficient sample size for significance, and use out-of-sample data to evaluate any parameters that you might fit if you use a structural model.
Some useful references:
A model is one which has the ability to make predictions. So use the models you have to make predictions about volatility. The one which can predict the best has to be the best (don't think you have a choice in that). If the predictability is comparable, then use any one. If you cannot test which one is the predicting the best, then go back and study what is volatility.
This is the edit after the first comment. For all volatility modelers, please read this paper first We Don't Quite Know What We are Talking About When We Talk About Volatility. Next it is important to understand that the process of volatility modeling is highly dependent on the context in which the volatility is going to be used in. There is no universal right answer.
For example, what do you believe in, price volatility or returns volatility and both have their space in financial research. next what time frame are you looking at, is it high frequency data ? or just intraday ? or end of day data. furthermore what do you want to predict using the model, because volatility itself is an invisible variable and hence you have to define some aspect of the market which you want to predict using your volatility model. So it could be to model the risk you face in the next period, or the prediction of the option price or simply the range of the market in the next period and believe me each of this is different. So using the model you would want to predict the measurable aspect of the market and then decide if the volatility model suits you or not. finally you have to understand if the model explains features of a volatility process the best, like regimes in volatility and spillovers to other instruments and the like.
This is not a question answered so easily. Keep working.