If $Y_t=\sum_{i=0}^qa_iX_{t-i}$ where $X_{t-i}$ is Gaussian with mean $\mu$ and variance $\sigma^2$, how do I show that $Y_t$ and $Y_{t+h}$ are independent (for $|h|>q$) using the joint pdf. I know this question has been asked before (here) but the answer is not what I'm looking for since independence is not shown, but only that $Y_t$ is Gaussian. I want to show independence between $Y_t$ and $Y_{t+h}$ by writing their joint pdf as the product of their marginal pdf's.

  • 5
    $\begingroup$ Hi: What you say is only true if $h > q$. Is that condition given ? If not, then they are not independent. $\endgroup$
    – mark leeds
    Commented Apr 21, 2021 at 4:15
  • $\begingroup$ @markleeds: Yes, it is given! Sorry I'll edit! $\endgroup$
    – Parseval
    Commented Apr 21, 2021 at 8:25
  • 1
    $\begingroup$ The way you want to do it seems complex to me. It's easier to just show that $cov(Y_t, Y_{t+h}) = 0$.Next we know that marginal distributions of $Y_t,$ and $Y_{t+h}$ are gaussian, Then you can use a theorem that says that, if the covariance of two normally distributed rv's is zero, then they are independent. Let me know if you need more help. $\endgroup$
    – mark leeds
    Commented Apr 21, 2021 at 17:36
  • $\begingroup$ @markleeds: Thank you very much for this answer! I'll try your method now and get back to you. $\endgroup$
    – Parseval
    Commented Apr 21, 2021 at 17:40
  • $\begingroup$ @markleeds - I need some help. Stuck here: \begin{align} \rho_{Y_t}(h)&=\text{Cov}[Y_t,Y_{t+h}]=\mathbb{E}[Y_tY_{t+h}]-\mathbb{E}[Y_t]\mathbb{E}[Y_{t+h}]\\ &=\mathbb{E}\left[\left(\sum_{j=0}^{q}\theta_jX_{t-j}\right)\left(\sum_{i=0}^q\theta_iX_{t+h-i}\right)\right]-\mathbb{E}\left[\sum_{j=0}^{q}\theta_jX_{t-j}\right]\mathbb{E}\left[\sum_{j=0}^{q}\theta_jX_{t+h-j}\right]\\ &= \mathbb{E}\left[\left(\sum_{j=0}^{q}\theta_jX_{t-j}\right)\left(\sum_{i=0}^q\theta_iX_{t+h-i}\right)\right]-\mu^2\sum_{j=0}^q \\ &= \end{align} $\endgroup$
    – Parseval
    Commented Apr 21, 2021 at 19:24

2 Answers 2


Hi Parseval: Let me put an answer here and hopefully it's clear. I fixed it so that $\mu = 0 $ is not needed but of course, the independent Gaussian assumption is still needed.

(1) $Y_t = \sum_{i=0}^{q} a_{i} X_{t-i}$

(2) $Y_{t+h} = \sum_{j=0}^{q} a_j X_{t+h-j}$

$ h > q$.

Note that the last (earliest ) term in (2) is $a_{q} X_{t+h-q}$ and first (latest ) term in (1) is $a_{0} X_{t}$. Therefore, since $h > q$, none of the noise terms in $Y_t$ overlap with any of the the noise terms in $Y_{t+h}$.

Now, the usual definition of covariance, gives:

$Cov(Y_{t}, Y_{t+h}) = E(\sum_{i=0}^{q} a_{i} X_{t-i} \sum_{j=0}^{q} a_{j} X_{t+h-j}) - E(\sum_{i=0}^{q} a_{i} X_{t-i}) E(\sum_{j=0}^{q} a_j X_{t+h-j}) $

So, for the first term we have the two sums multiplying each other and then an expectation is taken. Then, for the second term, we have two expectations multiplying each other.


Taking the expectations of the terms, we get $\sum_{i=0}^{q} a_{i} \mu \sum_{j=0}^{q} a_j \mu = \mu^2 \sum_{i=0}^{q} a_{i} \sum_{j=0}^{q} a_j$.

Simplifying this, results in:

$ \mu^2 \sum_{i=0}^{q} \sum_{j=0}^{q} a_i a_j $


$ E(\sum_{i=0}^{q} a_i X_{t-i} \sum_{j=0}^{q} a_j X_{t+h-j}) = $

$( \sum_{i=0}^{q} \sum_{j=0}^{q} a_{i} a_j ) E(X_{t-i} X_{t+h-j})$

But we showed earlier than that none of the terms in the very last expectation overlap, and, since they are independent Gaussians, we can re-write the last expression as $ E(X_{t-i} X_{(t+h-j)}) = E(X_{t-i}) E(X_{(t+h-j)}) = \mu^2$.

So, we showed that the first term equals the same thing as the second term which means that $Cov(Y_{t}, Y_{t+h}) = 0 $.

Note though that I needed the assumption that the X_{t} are independent Gaussian RV's but this is the usual assumption in the MA(q) model.

  • $\begingroup$ Hello Mark! Well the problem clearly specifies that $X\sim\text{WN}(\mu,\sigma^2)$ which is weird :S. I have a question regarding why you set $a_{j+h}$ for $Y_{t+h}$? Should't you only add index $h$ to the random variables? $a_i$ are constants. $\endgroup$
    – Parseval
    Commented Apr 22, 2021 at 9:51
  • $\begingroup$ Hi Parseval: I'm pretty sure that $\mu$ needs to be zero. I have to leave and will be gone for most of the day but I'll fix the notation when I get back tonight. As far as notation, I messed up and I should have written $(t +h - j) $ rather than $t-(j+h)$. Hopefully someone else can add to discussion and give insight on whether we need $\mu = 0$. I tried to work it out without that assumption and wasn't successful. Talk to you later. $\endgroup$
    – mark leeds
    Commented Apr 22, 2021 at 11:09
  • $\begingroup$ The mean of the $X_t$ is irrelevant, as if $\mu \ne 0$, subtract it out and adjust the mean values of the $Y_t$. Independence should not be orthogonal to the mean value. $\endgroup$ Commented Apr 22, 2021 at 17:44
  • $\begingroup$ You are assuming the $X_t$ are independent Gaussians. No further analysis should be required at that point, as $Y_t, Y_{t+h}$ are sums of non-intersecting independent Gaussians. $\endgroup$ Commented Apr 22, 2021 at 17:59
  • $\begingroup$ @steveo'america: I fixed it so that it just needed the non-intersecting independent Gaussians assumption and not the $\mu = 0$ assumption. Thanks for your help. Parseval: I fixed the notation also so hopefully it's clearer than it was previously. Thanks for pointing out the notation problems. $\endgroup$
    – mark leeds
    Commented Apr 22, 2021 at 18:05

If you are going to assume the $X_t$ are independent, then write $\vec{X}$ as the vector of $X_{t-q},\ldots,X_t,X_{t+h-q},\ldots,X_{t+h}$. Then $\vec{X} \sim \mathcal{N}\left(\mu \vec{1},\Sigma\right)$, where $\Sigma$ is a diagonal matrix. Then the vector of the two $Y$ values is $A \vec{X}$ for some block matrix that has one row of $q+1$ of the $a_i$, then $q+1$ zeros, then $q+1$ zeros and then the $a_i$ in the second row. Simple to confirm that $A\Sigma A^{\top}$ is diagonal.

  • $\begingroup$ We need not require all $X_t$ be independent, of course, only that the first $q+1$ elements of $\vec{X}$ be independent of the second $q+1$ elements, which would make $\Sigma$ block diagonal (2x2). $\endgroup$ Commented Apr 22, 2021 at 20:26
  • $\begingroup$ He did add later than it was an MA(q) so all the $X_{t}$ are independent. I like your proof with the block diagonal argument. It's more "visual" than ugly algebra. $\endgroup$
    – mark leeds
    Commented Apr 23, 2021 at 13:20
  • $\begingroup$ Parseval: Even with the independence assumption, you still need the $h > q$ assumption, Otherwise, if $h$ was not necessarily greater than $q$ as steveoamerica pointed, you would have some terms that were the same in the two $Y_t$ and $Y_{t+h}$ which makes $Y_t$ and $Y_{t+h}$ "dependent" because they contain some of the same terms. But, "dependent" is maybe a misleading term in that case. It's more like "common term" dependent. $\endgroup$
    – mark leeds
    Commented Apr 23, 2021 at 13:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.