3
$\begingroup$

This question has two parts, What is the state of the art in an academic or public knowledge sense of volatility forecast model evaluation?

Since there are many methods out in the wild, and do correct me if I am wrong but I haven't been able to find a complete list or any list for that matter of methods used so I'd like to start that here. It is very popular to use many of the different loss functions with arma-garch style models.

Some methods commonly used include:

Mean Absolute Error

$MAE = n^{-1} \sum_{t=1}^n | \sigma_t - h_t|$

$ \boldsymbol{R^2 \ log}$

$R^2LOG = n^{-1} \sum_{t=1}^n (log(\sigma_t^2 h_t^{-2}))^2 $

Improve this question
$\endgroup$
2
  • $\begingroup$ Possible duplicate of quant.stackexchange.com/questions/8056/… $\endgroup$
    – Jase
    Commented Oct 23, 2013 at 17:39
  • $\begingroup$ @Jase I guess the wording makes it seem similar, but this is more specific i'll update $\endgroup$
    – pyCthon
    Commented Oct 23, 2013 at 23:27

1 Answer 1

Reset to default
1
$\begingroup$

In Forecasting Financial Market Volatility Ser-Huang Poon dedicated entire chapter to the question, so the issue is far from simple. I don't believe there one single best way because of many questions that depend on model form and application such as

Should one evaluate volatility or variance, or perhaps ln(vol)? What is the benchmark - volatility is latent, should one use implied, or realized, and which realized measure should be used? What penalty should be used? L1 / L2? Points or percents? In addition there are specific tests to compare one model again another, s.a. Diebold & Mariano tests.

Usually what I see in academic literature is several metrics, L1 and L2 norm on volatility, as well as standard Mincer-Zarnowitz regression. In practical applications I most often see L1 on volatility, volatility %, or Vega * volatility.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Mincer-Zarnowitz is the standard for evaluating out of sample , how ever the $R^2$ based evaluation isn't suitable for comparison among different volatility forecast models. example, good $R^2$ and poor biased forecast.. $\endgroup$
    – pyCthon
    Commented Oct 24, 2013 at 2:10
  • $\begingroup$ I've dug deeper and it seems $R^2log$ maybe something interesting $\endgroup$
    – pyCthon
    Commented Oct 24, 2013 at 2:18
  • $\begingroup$ the Diebold & Mariano tests look very interesting thanks $\endgroup$
    – pyCthon
    Commented Oct 24, 2013 at 2:21
  • $\begingroup$ I assume if you have test your model with M-Z, and find it to be highly biased, you would do something about it in practice ... I think R^2 is more informative than you suggest. $\endgroup$
    – onlyvix.blogspot.com
    Commented Oct 25, 2013 at 1:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.