3
$\begingroup$

I have my estimates for an AR(3). To obtain the residuals I'm supposed to use $$Y_t-\hat\phi_0-\hat\phi_1Y_{t-1}-\hat\phi_2Y_{t-2}-\hat\phi_3Y_{t-3},$$

where the Y's are from the dataset. If I do this, won't I get a smaller number of residuals than observations?

I ask this, because in a book I'm reading, the authors fit an AR(3) from a dataset, and then present a plot of standardized residuals with the same numbers of observations. How is this possible?

Also, how does one obtain a standardized residual from a residual?

Improve this question
$\endgroup$
1

1 Answer 1

Reset to default
5
$\begingroup$

Usually for MLE estimation as you said we compute the residuals starting from index number of lag+1 (p+1 for AR model) in this case we obtain Conditional MLE estimates:

$\hat{\theta} = \text{arg max} \sum_{p+1}^{T} \ln f(Y_{t}|\theta)$

where $f(Y_{t})$ is the marginal density of observation $Y_{t}$ and $\ln$ is employed to maximized the log likelihood. The first residuals are then fixed as missing values and then we get a smaller number of usable residuals than observations. (in this case you start the recursion algorithm from index P+1)

However, it is also possible to estimate your model with Exact MLE estimates:

$\hat{\theta} = \text{arg max} \sum_{p+1}^{T} \ln f(Y_{t}|\theta) + \ln f(Y_{p},...,Y_{1},\theta)$

This require to fix some pre-sample values (the $Y_{0},...Y_{-p+1}$ ) in order to be able to run the model. Given that the AR model is stationary you can fix these values to their sample mean or unconditional theoretical mean. In this case first residuals are not missing values and you obtain the same number of usable residuals than observations. (in this case you start the recursion algorithm from index 1)

Regarding standardized residuals $res_{std}$, it is simply the residuals from the model divided by the conditional standard deviation : $ res_{std}= res /\sigma_{t}$ , this require to estimate $\sigma_{t}$ via, for instance, a GARCH model. If you don't model the conditional variance part, you use : $res_{std}= res /\sigma $ where $\sigma$ is the unconditional volatility of residuals obtained during estimation.

See this nice paper for details.

EDIT

I found my answer no enough documented so this is an update. First the MLE method for a simple AR1 model is very well explained in the following page

See below,

(from : Hamilton, J. D. (1994). Time Series Analysis (1 edition). Princeton University Press. (page 122) ):

enter image description here enter image description here enter image description here

[...]

enter image description here

So as we see the Exact Log Likelihood [equation 5.2.9] differs from the Conditional Log likelihood [equation 5.2.27] in the sense that in the exact MLE we maximize the full likelihood (that is why we call it exact) whereas in the conditional version we truncate the likelihood by dropping the marginal : $-0.5 \log(2\pi) -0.5\log(\sigma^{2}/(1-\phi^{2}))- \frac{(y_{1}-(c/1-\phi))^2}{2\sigma^{2}/(1-\phi^2}$

This marginal replaces the expectation of $E(Y_{0}) = c+\phi Y_{-1} + \epsilon$ by the unconditonal mean $E(Y_{0}) = \frac{c}{1-\phi}$

Improve this answer
$\endgroup$
4
  • $\begingroup$ I doubt that exact maximum likelihood estimator takes arbitrary initial values (such as the sample mean) rather than treating them as parameters to be estimated. Otherwise why would it be called "exact"? $\endgroup$
    – Richard Hardy
    Commented May 21, 2016 at 20:28
  • $\begingroup$ We do not take arbitrary initial values: sample mean should correspond to the theoritical mean for stationnary process. We call it exact MLE because it is based on the full likelihood and not on a truncated likelihood as it is for the Conditional MLE method .(conditioned on the first observation). + Numerically it will be very difficult to treat inital observations as parameters because I doubt that the likelihood function will converge. $\endgroup$
    – Malick
    Commented May 22, 2016 at 14:27
  • $\begingroup$ @Malick, Hmm, if that is true, then I have learnt something new today. Although I do not see how the exact ML is more exact than conditional ML if it works the way you say it works. But maybe that's just my problem. Thanks for you response. $\endgroup$
    – Richard Hardy
    Commented May 22, 2016 at 19:01
  • $\begingroup$ @RichardHardy see my edit $\endgroup$
    – Malick
    Commented May 25, 2016 at 17:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.