# evaluation of volatility models using loss functions

This question has two parts, What is the state of the art in an academic or public knowledge sense of volatility forecast model evaluation?

Since there are many methods out in the wild, and do correct me if I am wrong but I haven't been able to find a complete list or any list for that matter of methods used so I'd like to start that here. It is very popular to use many of the different loss functions with arma-garch style models.

Some methods commonly used include:

Mean Absolute Error

$MAE = n^{-1} \sum_{t=1}^n | \sigma_t - h_t|$

$\boldsymbol{R^2 \ log}$

$R^2LOG = n^{-1} \sum_{t=1}^n (log(\sigma_t^2 h_t^{-2}))^2$

-
Possible duplicate of quant.stackexchange.com/questions/8056/… – Jase Oct 23 '13 at 17:39
@Jase I guess the wording makes it seem similar, but this is more specific i'll update – pyCthon Oct 23 '13 at 23:27

## 1 Answer

In Forecasting Financial Market Volatility Ser-Huang Poon dedicated entire chapter to the question, so the issue is far from simple. I don't believe there one single best way because of many questions that depend on model form and application such as

Should one evaluate volatility or variance, or perhaps ln(vol)? What is the benchmark - volatility is latent, should one use implied, or realized, and which realized measure should be used? What penalty should be used? L1 / L2? Points or percents? In addition there are specific tests to compare one model again another, s.a. Diebold & Mariano tests.

Usually what I see in academic literature is several metrics, L1 and L2 norm on volatility, as well as standard Mincer-Zarnowitz regression. In practical applications I most often see L1 on volatility, volatility %, or Vega * volatility.

-
Mincer-Zarnowitz is the standard for evaluating out of sample , how ever the $R^2$ based evaluation isn't suitable for comparison among different volatility forecast models. example, good $R^2$ and poor biased forecast.. – pyCthon Oct 24 '13 at 2:10
I've dug deeper and it seems $R^2log$ maybe something interesting – pyCthon Oct 24 '13 at 2:18
the Diebold & Mariano tests look very interesting thanks – pyCthon Oct 24 '13 at 2:21
I assume if you have test your model with M-Z, and find it to be highly biased, you would do something about it in practice ... I think R^2 is more informative than you suggest. – onlyvix.blogspot.com Oct 25 '13 at 1:03