# Why is $Z_t$ uncorrelated with $X_{t-1}$ in $X_t=\theta X_{t-1}+Z_t$?

In a solution to the problem below, the teaching assistant solves it by calculating $$\mathbb{E}[X_t^2]$$ and ends up with also having to calculate $$\mathbb{E}[X_{t-1}Z_t]$$ after expanding the square. To do this, he states that "$$\mathbb{E}[X_{t-1}Z_t]=0$$ since $$X_{t-1}$$ is independent of $$Z_t$$ and $$X_{t-1}$$ is uncorrelated with $$Z_t$$".

Questions:

1. Why is $$X_{t-1}$$ independent of $$Z_t$$? I don't see that we assume that the time series is causal.
2. Why is $$X_{t-1}$$ uncorrelated with $$Z_t$$?

Problem:

Let a timeseries model $$X:=(X_t, t\in\mathbb{Z})$$ be given by

$$X_t=\phi X_{t-1}+Z_t, \quad \text{where} \quad Z_t\sim \text{WN}(0,\sigma^2)\quad \text{and} \quad |\phi|\neq 1.$$

Assume that the stochastic process satisfying this model is stationary. Compute the variance of $$X.$$

Note: Yes, I know that one simply can calculate $$\text{Var}[X_t]$$ directly in one line, but I'm trying to understand the motivations behind the instructors steps.

• Isn't this just the definition of the white noise $Z_t$? A new shock that is unrelated to the previous value of the process. Because of the normal distribution, uncorrelated = independent. Aug 19 at 14:12
• @Kevin in the non-causal case (i.e. $\phi > 1$), the stationary solution of the process is given by $X_t = - \sum_{j=1}^\infty \phi^{-j} Z_{t+j}$. I don't think it's obvious in this case that $E(X_t Z_{t+1}) = 0$ Aug 19 at 15:51
• @Kevin: White noise is not nessecarily normally distributed. I don't see how you concluded it to be normal. Aug 19 at 15:59
• @Parseval Good point, I tend to implicitly take white noise to be Gaussian. Nonetheless, to compute $\mathbb{V}\text{ar}[X]$ and $\mathbb{E}[XY]$, you only care about correlations (or covariances). Independence is stronger and not required (but equivalent for Gaussian white noise). The key remains: $Z_t$ is, by definition, uncorrelated to $X_{t-1}$. Aug 19 at 16:04

$$E(X_{t-1}Z_t) = 0$$ in the causal case $$|\phi | < 1$$, but not in the non-causal case $$|\phi | >1$$.

Causal case $$(|\phi| < 1)$$

In this case, the unique stationary solution to the AR(1) equation is given by $$X_t = \sum_{j=1}^\infty \phi^j Z_{t-j}$$ Thus, $$E(X_{t-1} Z_t) = E \left( Z_t \sum_{j=1}^\infty \phi^j Z_{t-1-j} \right) = \sum_{j=1}^\infty \phi^j E (Z_t Z_{t-1-j}) = 0$$ where the last equality follows from $$Z_i$$ and $$Z_j$$ being uncorrelated for $$i \neq j$$, and all the exchanges of integration are guaranteed by Fubini's theorem.

Non-causal case $$(|\phi| > 1)$$

The unique stationary solution to the AR(1) equation is $$X_t = - \sum_{j=1}^\infty \phi^{-j} Z_{t+j}$$ This equation can be arrived at by performing the recursion forward, rather than backward. Thus,

$$E(X_{t-1}Z_t) = -\sum_{j=1}^\infty \phi^{-j} E(Z_{t-1+j} Z_t) = - \frac{\sigma_Z^2}{\phi}$$

This finding is confirmed if you use the AR equations to get $$Var(X_t) = \phi^2 Var(X_{t-1}) + Var(Z_t) + 2 \phi E(X_{t-1} Z_t)$$

Using $$Var(X_{t-1}) = \frac{\sigma_Z^2}{\phi^2 - 1}$$ you get $$E(X_{t-1} Z_t) = - \frac{\sigma_Z^2}{\phi}$$.

Aside: most people restrict the study of ARMA processes to the first case for two reasons: (i) the non-causal future-dependent case is strange, and (ii) in the non-causal stationary case you may always find a white noise process $$\tilde{Z}_t$$ such that $$X_t$$ is the causal solution to $$X_t = \phi^{-1} X_{t-1} + \tilde{Z}_t$$. This may explain why your TA simply assumed $$E(X_{t-1}Z_t) = 0$$ always.

Edit: There seems to be some confusion in the comments about what the solutions to a time series equation are. I find most textbooks skim over this technical detail.

A solution to a stochastic equation $$g(X_t, Z_t) = 0$$ is a pair $$(X_t, Z_t)$$ such that $$(Z_t)_{t \in \mathbb{Z}}$$ is a white noise and for each $$t \in \mathbb{Z}$$, $$X_t$$ is a (measurable) function of the entire white noise sequence (i.e. $$X_t = h((\epsilon_{k})_{k\in\mathbb{Z}})$$) and $$X_t$$ satisfies the stochastic equation in some sense (e.g. almost surely). Note that $$X_t$$ can depend on the "past" noise, on the "future" noise, or the entire sequence restrictions. The solution $$(X_t, Z_t)$$ is said to be stationary if $$X_t$$ is a stationary process.

Notice that for the AR(1) equation $$X_t = \phi X_{t-1} + Z_t$$ we can find unique stationary solutions when $$|\phi| \neq 1$$; you can do this by recursion into the past if $$|\phi| < 1$$ or by recursion into the future (time-reversal) if $$|\phi| >1$$. See Example 3.1.2 and Theorems 3.1.1-3.1.3 of Brockwell and Davis for more details.

In the case $$|\phi| > 1$$, we may also fix a stochastic process $$(X_t)_{t \in \mathbb{N}}$$ such that $$X_0 = x$$ and for $$t \geq 1$$, $$X_t = X_0 + \sum_{j=1}^t Z_j$$. While this is indeed a solution to the AR(1) equation, it is clearly not stationary. Thus, we can remove this case, as the OP asked us to consider the solution to the AR(1) equation when $$X_t$$ is stationary.

• Hi. In the question, it says that the process satisfying the model is stationary. My textbooks have always said that $\phi > 1$ is a non-stationary process. I think non-causal means that the time is reversed which is not the case in this formulation. Aug 19 at 19:21
• Hi @markleeds, your textbooks seem to differ from mine. In particular, $\phi > 1$ falls under the stationary case (see e.g. Chapter 3 in Brockwell and Davis or Chapter 3 in Shumway and Stoffer). This is a technical point which is often overlooked in many other texts (with good reason, as they often focus on the causal---or past-dependent--- case). See my edit for more details on this. Aug 19 at 21:15
• Hi Jose: I have those texts but I can't say that they are my favorites. This question was cross posted to cross-validated so, if you care to, check out the answers over there. Dilipe Sawarte gave an answer which is more in line what I said. Maybe there's a terminology ambiguity when it comes to causal and non-causal. I'll check out those texts that you mentioned. Also, I'll try to find the cross-validated thread and send the link in another comment. Aug 20 at 4:01
• Here's the thread on cross-validated in case you are interested. stats.stackexchange.com/questions/540769/… Aug 20 at 4:02
• Hi Jose: It's probably best just to leave it as a terminology issue unless someone else throws in their 2 cents. I don't remember where I read it but, for me, causal versus non-causal always pertained to whether the response came before regressors ( in time ) or before. No problem and good to meet you. Aug 20 at 10:34