Help understanding the step $\sum_{j=0}^n\sum_{k=0}^ng_jg_k\text{Cov}(\epsilon_{n-1},\epsilon_{n+h-k})=\sum_{j=0}^ng_j^2+h\sigma^2$

Question

Given is that $\epsilon_n$ is a white noise process with $\text{Var}(\epsilon_n)=\sigma^2$ and that $g_j\in\mathbb{R}$. There is a step in my lecture notes that I don't get. It says the following

$$\sum_{j=0}^n\sum_{k=0}^ng_jg_k\text{Cov}(\epsilon_{n-j},\epsilon_{n+h-k})=\sum_{j=0}^ng_j^2+h\sigma^2 \quad \text{for} \quad h\ge0,$$

with the motivation "need $k=h+j$ otherwise the covariance is zero, we use this to remove the sum over $k$".

I understand that the sum over $k$ gets removed and that we want to avoid zero covariance, but how does $+h\sigma^2$ pop up? Doing the substitution $k=h+j$ then the covariance is just the variance which is onl $\sigma^2$ (no $h$), and then it's multiplied to the sum and not added.

Answer 1

$${\rm Cov} (\epsilon_{n-j}, \epsilon_{n+h-k}) =\gamma (h-k+j) = \sigma^2 1_{k=h+j} $$

where $1_A$ is the indicator function, set to $1$ if statement $A$ is true, and set to $0$ otherwise.

So the double summation is:

$$ \sum_{j=0}^\infty\sum_{k=0}^{\infty}g_jg_k\text{Cov}(\epsilon_{n-j},\epsilon_{n+h-k})$$ $$ = \sum_{j=0}^{\infty}\sum_{k=0}^{\infty} g_jg_k \sigma^2 1_{k=h+j} $$ $$= \sum_{j=0}^{\infty} g_jg_{h+j} \sigma^2 $$

as

$$ \sum_{k=0}^{\infty} g_jg_k \sigma^2 1_{k=h+j} = g_jg_{h+j} \sigma^2 $$

As $k$ runs from $0$ to $\infty$, the terms in the last summation are $0$ except when $k$ hits $h+j$, which will happen for any given, but fixed, $h$.

If the summation is finite, $k$ runs from $0$ to $n$, then

$$ \sum_{k=0}^{n} g_jg_k \sigma^2 1_{k=h+j} = g_jg_{h+j} \sigma^2 1_{h+j \leq n}$$