1
$\begingroup$

The question popped up when I was reading these lecture notes online. Consider the MA$(1)$ process given by $X_t=W_t+bW_{t-1}$ where $W_t$ is white noise distributed with constant variance $\sigma_W^2.$

They then proceed to calculate autocovariance function by

\begin{align} \gamma(k) &=E[X_tX_{t-k}]=E[(W_t+bW_{t-1})(W_{t-k}+bW_{t-k-1})]\\ &=E[W_tW_{t-k}]+E[bW_tW_{t-k-1}]+E[bW_{t-1}W_{t-k}]+E[b^2W_{t-1}W_{t-k-1}]. \end{align}

For $k=0$ we get the variance by $\gamma(0)=\text{Cov}[X_t,X_t]=\text{Var}[X_t]=\sigma_X^2,$ which expressed in terms of $\sigma_W^2$ becomes

\begin{align} \gamma(0) &=E[W_t^2]+bE[W_tW_{t-1}]+E[bW_{t-1}W_{t}]+b^2E[W_{t-1}^2]\\ &=\sigma^2_W+0+0+b^2\sigma^2_W = (1+b^2)\sigma_W^2. \end{align}

But the above only holds if the mean of the process $W$ is equal to zero.

Question 1: Why is $E[W_tW_{t-1}]=0?$

Question 2: What if $W\sim\text{WN}(\mu,\sigma_W^2)?$ That is, how would the calculations change?


Here is my attempt: If $\mu \neq 0$ then $E[W_t^2]=\text{Var}[W_t]+E[W_t]^2=\sigma_W^2+\mu^2$. If it's still true that $E[W_tW_{t-1}]=0$ then we obtain $\gamma(0)=\sigma^2_W+\mu^2+b^2(\sigma^2_W+\mu^2)=(\sigma^2_W+\mu^2)(1+b^2).$

$\endgroup$
4
  • 4
    $\begingroup$ E[W(t) W(t-1)] is zero because white noise terms are zero mean and uncorrelated. If they are no longer zero mean, then this does not hold any more, it will be non-zero. $\endgroup$ – noob2 Apr 20 at 15:27
  • $\begingroup$ @noob2 - Thank you. But do you know how to calculate $E[W_tW_{t-1}]$ to know exactly which non-zero value it will be? $\endgroup$ – Parseval Apr 20 at 15:31
  • $\begingroup$ By a reasoning very similar to what you already did, I believe it will be $\mu^2$, but check the math, I may be wrong. $\endgroup$ – noob2 Apr 20 at 16:13
  • 1
    $\begingroup$ Writing $W_t = \mu + Z_t$ where $Z_t$ are zero mean, independent Gaussians, then using 'FOIL', you should get an expected value of $\mu^2$. $\endgroup$ – steveo'america Apr 20 at 16:22
3
$\begingroup$

I feel that question 1 has already been answered in the comments, so I will provide a brief answer and insight for question 2 (which is also your title question).

Question 2: What happens when $W \sim N(\mu, \sigma^2)$?

You will get a moving average with a specific intercept-term. The intercept does not have any impact on the autocovariances and thus they will stay the same as the original moving average. To see this, let $\bar{W} \sim N(\mu,\sigma^2)$ be a white-noise process with constant drift. Then, we can quickly observe that:

\begin{align*} X_t &= \bar{W}_t + b \bar{W}_{t-1}\\ &= (\mu + W_t) + b \cdot (\mu + W_{t-1})\\ &= \mu + b \mu + W_t + bW_{t-1}\\ &= \nu + W_t + bW_{t-1}, \end{align*} with $\nu = \mu + b\mu$ and $W_t \sim N(0, \sigma^2)$ be an uncorrelated white-noise process. Evidently, we also get that $\mathbb{E}\left[X_t\right]=\nu$. Now, we have a MA(1) process with constant drift, and therefore we cannot use the equation on page 3 in the slides (as seen in your question), since the author assumes $X_t$ is a zero-mean process. Instead, taken directly from the definition of autocovariance, it is observed that the mean does not have any impact on the autocovariances:

\begin{align} \gamma(h)&=\mathbb{C}ov\left(X_t,X_{t-h}\right)\\ &=\mathbb{C}ov\left(\nu + W_t + bW_{t-1}, \: \nu + W_{t-h} + bW_{t-h-1}\right)\\ &=\mathbb{C}ov\left(W_t,W_{t-h}\right) + b\mathbb{C}ov\left(W_t,W_{t-h-1}\right) + b\mathbb{C}ov\left(W_{t-1},W_{t-h}\right) + b^2\mathbb{C}ov\left(W_{t-1},W_{t-h-1}\right),\\ \end{align}

where we have used covariance properties of linear combinations. Therefore, we get the same results found in the slides for all $h$:

\begin{equation} \gamma(h) = \begin{cases} (1+b^2)\sigma^2 & \text{if } h = 0 \\ b\sigma^2 & \text{if } h=1\\ 0 & \text{if } h\geq 2 % \end{cases} . \end{equation}

If you want to verify this yourself, then you could proceed as done in the slides, however, the formula on page 3 for $X_t$ being a process with non-zero mean, can be written as (directly from the definition of (auto-)covariance):

\begin{equation} \gamma(h)=\mathbb{C}ov\left(X_t,X_{t-h}\right) = \mathbb{E}\left[X_t X_{t-h}\right] - \mathbb{E}\left[X_t\right] \mathbb{E}\left[X_{t-h}\right]. \end{equation}

Simulating the results:

As a quick conclusion, we can verify the results by simulating a MA(1) process with drift (this is done in R) and see whether the mean of the process is equal to $\nu$ and whether the autocorrelations (for $h\geq 2$) converge towards zero. Let, $b=0.5, \: \mu=10$ and $\sigma^2 = 1$, then $\nu=15$ and can be verified by running the following code:

MA_process_sim <- function(b, mu, sigma, Nmesh){

  X <- numeric()

  W <- rnorm(Nmesh+1, mu, sigma)

  X[1] <- 0

  for(i in 1:Nmesh){

    X[i+1] <- W[i+1] + b*W[i]
    
  }

  return(X[-1])

}

test <- MA_process_sim(0.5, 10, 1, 425000)

mean(test)
acfs <- acf(test, plot = F)

with mean(test) giving us 14.9979. The acfs for $h=1$ results in $0.4$, which is the same as the theoretical result, $\rho(1)=\frac{\gamma(1)}{\gamma(0)}= \frac{b\sigma^2}{(1+b^2)\sigma^2}= 0.4$. Furthermore, for $h \geq 2$ the acf-values are very close to zero. I hope this provide some help and insight.

Additional picture of simulation results:

results

$\endgroup$
2
$\begingroup$

(Q2) Adding a constant $\mu$ to the white noise, we have

$$Y_t := (\mu+W_t) + b(\mu + W_{t-1}) = \mu(1+b) + X_t, $$

which is MA(1) with drift:

$$E[Y_t] = \mu(1+b) $$

and

$$ {\rm Cov}(Y_t, Y_{t-k}) = {\rm Cov}(\mu(1+b) + X_t, \mu(1+b) + X_{t-k}) $$ $$ = {\rm Cov}(X_t, X_{t-k})$$

(last equality coming from general covariance properties).

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.