1
$\begingroup$

In a solution to the problem below, the teaching assistant solves it by calculating $\mathbb{E}[X_t^2]$ and ends up with also having to calculate $\mathbb{E}[X_{t-1}Z_t]$ after expanding the square. To do this, he states that "$\mathbb{E}[X_{t-1}Z_t]=0$ since $X_{t-1}$ is independent of $Z_t$ and $X_{t-1}$ is uncorrelated with $Z_t$".

Questions:

  1. Why is $X_{t-1}$ independent of $Z_t$? I don't see that we assume that the time series is causal.
  2. Why is $X_{t-1}$ uncorrelated with $Z_t$?

Problem:

Let a timeseries model $X:=(X_t, t\in\mathbb{Z})$ be given by

$$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim \text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$

Assume that the stochastic process satisfying this model is stationary. Compute the variance of $X.$

Note: Yes, I know that one simply can calculate $\text{Var}[X_t]$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.

$\endgroup$
4
  • 5
    $\begingroup$ Isn't this just the definition of the white noise $Z_t$? A new shock that is unrelated to the previous value of the process. Because of the normal distribution, uncorrelated = independent. $\endgroup$
    – Kevin
    Aug 19 at 14:12
  • $\begingroup$ @Kevin in the non-causal case (i.e. $\phi > 1$), the stationary solution of the process is given by $X_t = - \sum_{j=1}^\infty \phi^{-j} Z_{t+j}$. I don't think it's obvious in this case that $E(X_t Z_{t+1}) = 0$ $\endgroup$ Aug 19 at 15:51
  • $\begingroup$ @Kevin: White noise is not nessecarily normally distributed. I don't see how you concluded it to be normal. $\endgroup$
    – Parseval
    Aug 19 at 15:59
  • 1
    $\begingroup$ @Parseval Good point, I tend to implicitly take white noise to be Gaussian. Nonetheless, to compute $\mathbb{V}\text{ar}[X]$ and $\mathbb{E}[XY]$, you only care about correlations (or covariances). Independence is stronger and not required (but equivalent for Gaussian white noise). The key remains: $Z_t$ is, by definition, uncorrelated to $X_{t-1}$. $\endgroup$
    – Kevin
    Aug 19 at 16:04
3
$\begingroup$

$E(X_{t-1}Z_t) = 0$ in the causal case $|\phi | < 1$, but not in the non-causal case $|\phi | >1$.

Causal case $(|\phi| < 1)$

In this case, the unique stationary solution to the AR(1) equation is given by $$X_t = \sum_{j=1}^\infty \phi^j Z_{t-j}$$ Thus, $$E(X_{t-1} Z_t) = E \left( Z_t \sum_{j=1}^\infty \phi^j Z_{t-1-j} \right) = \sum_{j=1}^\infty \phi^j E (Z_t Z_{t-1-j}) = 0$$ where the last equality follows from $Z_i$ and $Z_j$ being uncorrelated for $i \neq j$, and all the exchanges of integration are guaranteed by Fubini's theorem.

Non-causal case $(|\phi| > 1)$

The unique stationary solution to the AR(1) equation is $$X_t = - \sum_{j=1}^\infty \phi^{-j} Z_{t+j}$$ This equation can be arrived at by performing the recursion forward, rather than backward. Thus,

$$E(X_{t-1}Z_t) = -\sum_{j=1}^\infty \phi^{-j} E(Z_{t-1+j} Z_t) = - \frac{\sigma_Z^2}{\phi}$$

This finding is confirmed if you use the AR equations to get $$Var(X_t) = \phi^2 Var(X_{t-1}) + Var(Z_t) + 2 \phi E(X_{t-1} Z_t)$$

Using $Var(X_{t-1}) = \frac{\sigma_Z^2}{\phi^2 - 1}$ you get $E(X_{t-1} Z_t) = - \frac{\sigma_Z^2}{\phi}$.

Aside: most people restrict the study of ARMA processes to the first case for two reasons: (i) the non-causal future-dependent case is strange, and (ii) in the non-causal stationary case you may always find a white noise process $\tilde{Z}_t$ such that $X_t$ is the causal solution to $X_t = \phi^{-1} X_{t-1} + \tilde{Z}_t$. This may explain why your TA simply assumed $E(X_{t-1}Z_t) = 0$ always.


Edit: There seems to be some confusion in the comments about what the solutions to a time series equation are. I find most textbooks skim over this technical detail.

A solution to a stochastic equation $$g(X_t, Z_t) = 0$$ is a pair $(X_t, Z_t)$ such that $(Z_t)_{t \in \mathbb{Z}}$ is a white noise and for each $t \in \mathbb{Z}$, $X_t$ is a (measurable) function of the entire white noise sequence (i.e. $X_t = h((\epsilon_{k})_{k\in\mathbb{Z}})$) and $X_t$ satisfies the stochastic equation in some sense (e.g. almost surely). Note that $X_t$ can depend on the "past" noise, on the "future" noise, or the entire sequence restrictions. The solution $(X_t, Z_t)$ is said to be stationary if $X_t$ is a stationary process.

Notice that for the AR(1) equation $X_t = \phi X_{t-1} + Z_t$ we can find unique stationary solutions when $|\phi| \neq 1$; you can do this by recursion into the past if $|\phi| < 1$ or by recursion into the future (time-reversal) if $|\phi| >1$. See Example 3.1.2 and Theorems 3.1.1-3.1.3 of Brockwell and Davis for more details.

In the case $|\phi| > 1$, we may also fix a stochastic process $(X_t)_{t \in \mathbb{N}}$ such that $X_0 = x$ and for $t \geq 1$, $X_t = X_0 + \sum_{j=1}^t Z_j$. While this is indeed a solution to the AR(1) equation, it is clearly not stationary. Thus, we can remove this case, as the OP asked us to consider the solution to the AR(1) equation when $X_t$ is stationary.

$\endgroup$
6
  • 2
    $\begingroup$ Hi. In the question, it says that the process satisfying the model is stationary. My textbooks have always said that $\phi > 1$ is a non-stationary process. I think non-causal means that the time is reversed which is not the case in this formulation. $\endgroup$
    – mark leeds
    Aug 19 at 19:21
  • $\begingroup$ Hi @markleeds, your textbooks seem to differ from mine. In particular, $\phi > 1$ falls under the stationary case (see e.g. Chapter 3 in Brockwell and Davis or Chapter 3 in Shumway and Stoffer). This is a technical point which is often overlooked in many other texts (with good reason, as they often focus on the causal---or past-dependent--- case). See my edit for more details on this. $\endgroup$ Aug 19 at 21:15
  • $\begingroup$ Hi Jose: I have those texts but I can't say that they are my favorites. This question was cross posted to cross-validated so, if you care to, check out the answers over there. Dilipe Sawarte gave an answer which is more in line what I said. Maybe there's a terminology ambiguity when it comes to causal and non-causal. I'll check out those texts that you mentioned. Also, I'll try to find the cross-validated thread and send the link in another comment. $\endgroup$
    – mark leeds
    Aug 20 at 4:01
  • $\begingroup$ Here's the thread on cross-validated in case you are interested. stats.stackexchange.com/questions/540769/… $\endgroup$
    – mark leeds
    Aug 20 at 4:02
  • 3
    $\begingroup$ Hi Jose: It's probably best just to leave it as a terminology issue unless someone else throws in their 2 cents. I don't remember where I read it but, for me, causal versus non-causal always pertained to whether the response came before regressors ( in time ) or before. No problem and good to meet you. $\endgroup$
    – mark leeds
    Aug 20 at 10:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.