# GARCH(1,1) parameter estimation optimization method

In estimating a GARCH(1,1) model, $$\sigma_{t+1}^2 = \omega+\alpha \epsilon_t^2+\beta\sigma_t^2$$ Usually the parameter tuple $$(\omega,\alpha,\beta)$$ is estimated by the quasi-maximal likelihood. However, it seems hard to find the optimal parameter estimation stably. Are there any references for explicitly dealing with the optimization issue?

• Should the focus lay on stability of the coefficients or the optimization in general ?
– Lars
Jun 25 at 15:42
• @Jonas_Dim: Either would be nice.
– Hans
Jun 25 at 16:46
• Take a look at: "Analytic Derivatives and the Computation of Garch Estimates", Gabriele Fiorentini, Giorgio Calzolari and Lorenzo Panattoni Journal of Applied Econometrics 1996. They discuss the derivation of analytic derivatives for regression models with a GARCH error term. Also the computation of the hessian and information matrix are discussed in detail.
– Lars
Jun 25 at 19:30
• Genaro Sucarrat may have some discussion in the papers accompanying his R packages for GARCH estimation. garchx is a relatively new one. Jun 25 at 20:55
• @Hans, I do not immediately see whether this may work, and I cannot say much more without digging deep into the details. I also agree with the comments by Jonas_Dim. Jun 26 at 7:33

Let's say we have a time series $$\left\{\epsilon_t\right\}_{t=1}^T$$ of daily log-returns and we want to estimate the model: \begin{align} \epsilon_t&=\sigma_tu_t ,\quad u_t \overset{iid}{\sim}{\cal N}(0,1)\\ \sigma_t^2&=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1\sigma_{t-1}^2 \end{align} If I understood your idea right, then you want to estimate the regression: \begin{align} \epsilon_t^2=\alpha_0+(\alpha_1+\beta_1)\epsilon_{t-1}^2+\eta_t \end{align} However, I dont understand how you come up with this. In my opinion, you start with the conditional variance equation: $$$$\sigma_t^2=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1\sigma_{t-1}^2$$$$ Now add $$w_t=\epsilon_t^2-\sigma_t^2$$ on both sides. You obtain: \begin{align} &\sigma_t^2+w_t=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1\sigma_{t-1}^2+w_t \\ \leftrightarrow &\sigma_t^2+\epsilon_t^2-\sigma_t^2=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1\sigma_{t-1}^2+w_t\\ \leftrightarrow &\epsilon_t^2=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1\sigma_{t-1}^2+ w_t \\ \end{align} Notice that $$w_{t-1}=\epsilon_{t-1}^2-\sigma_{t-1}^2$$ and therefore $$\sigma_{t-1}^2=\epsilon_{t-1}^2-w_{t-1}$$. You obtain: \begin{align} \epsilon_t^2=\alpha_0+\alpha_1\epsilon_{t-1}^2+\beta_1(\epsilon_{t-1}^2-w_{t-1})+ w_t \\ \leftrightarrow \epsilon_t^2=\alpha_0+(\alpha_1+\beta_1)\epsilon_{t-1}^2 -\beta_1w_{t-1} +w_t \end{align} If $$E(\epsilon_t^4)<\infty$$ $$\left\{w_t \right\}$$ has finite variance and is a weak white noise, so the GARCH(1,1)-model has an ARMA(1,1)-representation for the squared returns. If I remember correctly, OLS for ARMA(1,1) is inconsistent and ML for this model seems to be difficult too. Even if we assume that $$u_t \overset{iid}{\sim}{\cal N}(0,1)$$ what is the conditional distribution of $$w_t$$ ? I have no idea what the analytic form of the likelihood would be.

It seems like is is possible to estimate the model this way but to be honest, I have never seen this approach before and I have the feeling that the results will be terrible. Maybe try it and then compare the results with the standard approach via ML ?

Basically, my approach would be:

1. Choose initial value $$\theta_0=(\alpha_0,\alpha_1,\beta_1)'$$.
2. Choose inital values for $$\epsilon_0^2$$ and $$\sigma_0^2$$, $$\epsilon_0^2=\sigma_0^2=\frac{1}{T}\sum_{t=1}^T\epsilon_t^2$$ is a natural choice.
3. For the given parameter vector $$\theta_i$$ calculate $$\sigma_t^2$$.
4. Use the results to calculate the log conditional densities $${\cal l}_t(\theta_i)=-\frac{1}{2}\ln(\sigma_t^2)-\frac{1}{2}\frac{\epsilon_t^2}{\sigma_t^2}$$
5. Use some optimization method, I think BHHH is often used. $$\theta_{i+1}=\theta_i+\lambda\left[\sum_{t=1}^T\frac{\partial{\cal l}_t(\theta_i)}{\partial \theta_i}\frac{\partial{\cal l}_t(\theta_i)}{\partial \theta_i'}\right]^{-1}\sum_{t=1}^T\frac{\partial{\cal l}_t(\theta_i)}{\partial \theta_i}$$
6. Stop if $$\vert\vert \theta_{i+1}-\theta_i\vert\vert<\epsilon$$, for example $$\epsilon=10e^{-4}$$. If not return to 3.

However, I am not an expert in optimization, it would be nice to hear other opinions on that.

• I will look into the question you raised. You mentioned in the comment below my answer that you would post "a way how I would approach this". Could you please post that? Thank you.
– Hans
Jun 26 at 18:46
• I edited my answer @Hans
– Lars
Jun 26 at 19:46
• Well, this is just one form of the Newton Ralphson method with the step size $\lambda$, right? OK, you call it BHHH. In these kind of algorithm, you have to find an initial point that is close enough. I am using scipy.optimize.minimize(). It has all these similar optimization algorithms. I am getting unstable/nonsensical result. That is why I posed the question. The only thing that may make a difference is I did not put in the explicit form of the first and second derivatives. Maybe I should try that first.
– Hans
Jun 26 at 20:47
• I think the Berndt–Hall–Hall–Hausman algorithm was actually the optimization method proposed by bollerslev in his original paper. Take a look at „Generalized autoregressive conditional heteroskedasticity“ (1986) Journal of Econometrics.
– Lars
Jun 26 at 21:21
• The hardest part is to pick the right initial point. Is there a paper showing whether the semi-log-likelihood of GARCH(1,1) is concave with respect to $(\omega,\alpha,\beta)$?
– Hans
Jun 26 at 23:18

Would it make sense to do the following?

Let the estimation of $$\sigma_t^2$$ be $$\epsilon_t^2$$ and $$\epsilon_{t+1}^2 = \omega+(\alpha+\beta)\epsilon_t^2+u$$ where $$u$$ is the residual. We apply linear regression to the above AR model and obtain the estimate of $$\omega$$ and $$\alpha+\beta$$. We only need to estimate one variable, either $$\alpha$$ or $$\beta$$. We do so through maximizing the quasi-maximal likelihood. This time, it is a $$1$$ as opposed to $$3$$ dimensional problem, which would be much easier.

• I dont think that this is a valid approach. In order to estimate $\sigma_t^2$ you need estimates for $\omega$, $\alpha$ and $\beta$. Basically, you choose starting values for $\sigma_0^2$ and $\epsilon_0^2$. Then you calculate $\sigma_1^2=\hat{\omega}+\hat{\alpha}\epsilon_0^2+\hat{\beta}\sigma_0^2$ and so on.
– Lars
Jun 26 at 6:26
• Also I am a bit confused by "we only need estimates for one variable, either $\alpha$ or $\beta$". It is possible to kick out $\omega$ of the equation by using variance targeting, i.e. replacing $\omega$ by $\omega=(\alpha+\beta)E(\epsilon_t^2)$ but we still need estimates for $\alpha$ and $\beta$.
– Lars
Jun 26 at 6:35
• @Jonas_Dim: 1) I am saying the equation I write out in my answer is a linear regression with intercept $\omega$ and slope $\alpha+\beta$. We can readily obtain these two values. 2) Should it not be $\omega = (1-\alpha-\beta)\mathbf E[\epsilon_t^2]$?
– Hans
Jun 26 at 14:10
• How do you get $\alpha$ and $\beta$ from $\alpha+\beta$? Jun 26 at 14:32
• @RichardHardy: My last sentence in my answer says we can do so via maximizing the quasi-maximal likelihood as usual. Only this time, it is a one-dimensional, as opposed to 3 dimensional, problem which is much easier to solve.
– Hans
Jun 26 at 14:34