# Deciding (p,q) in garch and model test on empirical data

I'm currently working on a dataset containing data from the 29 January till the 29 July 2009. In the dataset I have prices of the S&P 500 index for all days. Furthermore, I have the implied volatilities and prices of ATM option with 3 month to expiry.
I have read in Awartani and Corradi from 2005, that EGARCH(1,1) is particularly useful in estimating volatily from this index.
However, I want to test this statement and see if I can find a better model EGARCH(p,q) model. My setup in the expirement is as following: I have set up 20 non-overlapping portfolios (starting 125 days in the dataset) with 3 month to expiry options. From these 20 non-overlapping portfolios, I want to test what EGARCH(p,q) model is the best to forecast the portfolios volatilities (Delta-hedge portfolio).
I have 2 questions)

1. What is the best method in figuring out the (p,q). I have read that AIC or FIC are the best methods in doing so, can anybody elaborate if this is true? Or are easier methods preferred?
2. Deciding what (p,q) to choose, should I perform the test on the whole dataset, or fractions of the dataset. Example) The first portfolio starting the 28/7-2004, then find the optimal (p,q) from 29 January 2004 til 27 July 2004?

I will further elaborate on the answer of @Jonas_Dim.

### Choosing the GARCH-order:

It would be best to focus on AIC, BIC, and maximum log-likelihood, to compare in-sample model fits. In essence, BIC is less tolerant of free parameters (parameters to be estimated) for a high amount of in-sample data $$N$$, due to the virtue of how it penalize free parameters, ($$\ln(N)\cdot k$$, $$\: k$$ being number of free parameters). In some cases it might lead to a model that underfits the true data generating process. AIC does not tend to do that, but might under penalize free parameters leading to a selection of models that might overfit. This has been discussed meticulously on stats SE, and I advise you to take a look at some of the answers.

In general, you should not go over the GARCH-order of $$p=q=2$$, since it very often leads to overfitting and rarely performs great out-of-sample. This is further detailed in the famous paper, Does anything beat a GARCH(1,1)? where they state:

Restricting the models to have two lags (or less) should not affect the main conclusions of our empirical analysis, because it is unlikely that a model with more lags would outperform a simple benchmark in the out-of-sample comparison, unless the same model with two lags can outperform the benchmark. This aspect is also evident from our analysis, where a model with $$p=q=2$$ rarely performs better (out-of-sample) than the same model with fewer lags, eventhough most parameters are found to be signiﬁcant (in-sample).

I would recommend only to test a few EGARCH orders and then take the one that provides the best BIC/AIC (or both).

Regarding your second question, you usually choose an in-sample period where you estimate your models and evaluate them. You can then choose the model that minimizes one (or both) of the above criteria and go with that. As depicted in the answer of @Jonas_Dim, a good in-sample fit as seen by BIC/AIC does not imply good out-of-sample forecast performance.

• Thanks for the answers @Pleb and Jonas_dim, both are really helpfull! – Sebastian Strauss Hansen May 15 at 11:56
• I totally agree with @Pleb. I think the key question is whether the focus lays on in-sample fit or out-of-sample performance. Depending on that you should decide what method you use. However, it is a good idea to consider the GARCH(1,1) as the "standard" model and then check whether other models perform better. – Lars May 15 at 15:25

Regarding your first question, I think there are different methods:

1. If a good in-sample fit is important to you, you can look at AIC, BIC or maximized log-likelihood and then choose your model according to that.
2. However, a good in-sample fit does not mean that the model is good in predicting volatility. So if your focus lays on the volatility forecasts, it makes sense to compare the forecasts from different models with each other. I think a natural way to do this is to re-estimate your model daily, update the parameters and then predict the volatility for the next trading day until you have no observations left. However, it is hard to evaluate the forcasts. It is well known that squared daily returns are a very noisy proxy for the "true" unobserved volatility. So in the literature people use realized volatility measures. But you need high-frequency intraday data in order to compute these realized volatility measures and this adds a lot of complexity to your task (even getting the data is very difficult). Even if you have the data, you need to ask yourself what type of evaluation statistic you should use. For example, you could use the classic measures like Mincer-Zarnowitz regression, MSE or MAE, or even a funcion that penalties an underprediction of volatility more than an overprediction like QLIKE.

Long story short, I think it is really difficult to find the "right" model order (especially when it comes to the evaluation of forecasts) and from my experience in many cases a simple (1,1) model performs the best. So you could safe a lot of time and effort by simply using the order (1,1).