Sums of random variables and independence

Question

I'm having troubles with this proof:

Let $\{Z_i\}_{i\in\mathbb{Z}}$ be i.i.d. random variables with zero mean and unit standard deviation. For $(a_0, a_1, ..., a_r)$ a sequence of $r$ real numbers and $j\in\mathbb{Z}$, let

$$\begin{align} Y_j & = \sum_{i=0}^ r a_i Z_{j-i} \end{align}$$

For $r=2$, state and prove a general and sufficient condition on $(a_0, a_1, a_2)$ such that $Y_j$ and $Y_{j-1}$ are independent regardless of the probability distribution of $Z$.

How can one prove independence in this case? I was thinking about applying convolution and then expliciting $(a_0, a_1, a_2)$ in a way that $f_{Y_{j}, Y_{j-1}} = f_{Y_j}f_{Y_{j-1}}$ holds... But it seems to be a little complicated and I don't really know how to tackle the problem in a simpler way. Any help would be greatly appreciated!

Answer 1

Write out the simple equations

$$\begin{align} Y_j &= a_0 Z_j + a_1 Z_{j-1} + a_2 Z_{j-2}\\ Y_{j-1} &= a_0 Z_{j-1} + a_1 Z_{j-2} + a_2 Z_{j-3} \end{align}$$

There are some very simple cases that make $Y_j \perp Y_{j-1}$ due to the independence assumption of the random variables $\{Z_i\}_{i\in\mathbb{Z}}$. An example is $a_0 \in \mathbb{R}\setminus \{0\},\, a_1 = 0,\, a_2 = 0$. Not sure if you were looking for a complete solution but this should help get you started.

Also, an easy check for RV which are not independent is using the contrapositive form of the common theorem $$X\perp Y \implies E[XY] = E[X]E[Y]$$ Note that the converse of this statement is not true.

Proof

Assertion $a_1a_0 + a_2a_1 = 0 \iff Y_j \perp Y_{j-1}$

Define $\mu = E[Z]$

($\implies$) Suppose $a_1a_0 + a_2a_1 = 0$. There are two cases where this is possible. Case 1, suppose $a_1 = 0$. The equations become

$$\begin{align} Y_j &= a_0 Z_j + a_2 Z_{j-2}\\ Y_{j-1} &= a_0 Z_{j-1} + a_2 Z_{j-3} \end{align}$$

Their $\sigma$-algebras are given by $\sigma(Y_j) = \sigma(Z_j)\cup\sigma(Z_{j-2})$ and $\sigma(Y_{j-1}) = \sigma(Z_{j-1})\cup \sigma(Z_{j-3})$. Thus $Y_j \perp Y_{j-1}$. This could be more gruesomely detailed but I take some for granted. See this for more details including definitions etc.

Case 2, suppose $a_2 = 0$ and $a_0 = 0$. The equations become

$$\begin{align} Y_j &= a_1 Z_{j-1} \\ Y_{j-1} &= a_1 Z_{j-2} \end{align}$$

The same $\sigma$-algebra argument applies more easily but a more elegant solution presents itself in the form of the CDF.

$$\begin{align} F_{Y_j, Y_{j-1}}(y_j, y_{j-1}) &= P(Y_j \leq y_j \text{ and } Y_{j-1} \leq y_{j-1}) \\ & = F_{Z_j}(y_j/a_1)F_{Z_{j-1}}(y_{j-1}/a_1)\\ & = F_{Y_j}(y_j)F_{Y_{j-1}}(y_{j-1}) \end{align}$$

($\impliedby$) Suppose $Y_j \perp Y_{j-1}$ by theorem, we know that $E[Y_j Y_{j-1}] = E[Y_j]E[Y_{j-1}]$ calculating these values separately,

$$\begin{align} E[Y_jY_{j-1}] & = (a_0^2 + a_0a_1 + a_0 a_2 + a_1^2 + a_1 a_2 + a_2 a_0 + a_2^2 )\mu^2 \\ & + (a_1a_0 + a_2a_1)E[Z^2] \end{align}$$

$$\begin{align} E[Y_j]E[Y_{j-1}] & = (a_0^2 + a_0a_1 + a_0 a_2 + a_1^2 + a_1 a_2 + a_2 a_0 + a_2^2)\mu^2\\ &+ (a_1a_0 + a_2a_1)\mu^2 \end{align}$$

In the non-degenerate case when the distribution of $Z$ is not a constant, the variance is strictly positive so that $E[Z^2] - \mu^2 > 0$ and so $E[Z^2] > \mu^2$ and more importantly $E[Z^2] \neq \mu^2$ Thus for the equality $E[Y_j Y_{j-1}] = E[Y_j]E[Y_{j-1}]$ to hold, it must be the case that $a_1a_0 + a_2a_1 = 0$.

Answer 2

My first idea would be to try writing up the characteristic functions and use the theorem stated in the bottom answer about independence here: https://math.stackexchange.com/questions/376511/a-criterion-for-independence-based-on-characteristic-function