Let $$X_1\sim N(\mu_{X_1},\sigma_{X_2}^2)$$ $$X_2\sim N(\mu_{X_2}, \sigma_{X_2}^2)$$ where $\mu_{X_2}=c+aX_1$. Also, I have data $D$ (with missing values on $X_1,X_2$).

How can I update/estimate the parameters $\mu_{X_1},\sigma_{X_1},\mu_{X_2},a,c,\sigma_{X_2}$ using EM? i.e. what is the formula for updating $\sigma_{X_2}$?

My model is a conditional Gaussian, which is a conditional form of a bivariate Gaussian $(X_1,X_2)$ with mean vector $(\mu_1,\mu_2)^\top$ and covariance matrix $$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$

Here is a reference to convert bivariate Gaussian to conditional Gaussian: $$\mu_{2|1}=\mu_{2}+\Sigma_{21}\Sigma_{11}^{-1}(X_1-\mu_1)\quad,\quad \Sigma_{22|1}=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$$ which yields my model.

It seems that when $X_1$ has different observations and $X_2$ is unobserved, the variance of $X_2$ remains unchanged.So how to update the $\sigma_{x_2}$? Do I need to estimate covariance matrix?

Initial setting for the model $$X_1\sim N(5,7)\quad,\quad X_2\sim N(0.5X_1,8)$$



$$\quad X_2:\operatorname{NA\,\,NA\,\,3}$$

  • 2
    $\begingroup$ Your model is not clear. You may need to describe your data and provide more details on your model. $\endgroup$ – Gordon Jul 21 '16 at 0:55
  • $\begingroup$ revised as required. $\endgroup$ – snowave Jul 21 '16 at 9:52
  • $\begingroup$ This is still not clear, and you have too less data. $\endgroup$ – Gordon Jul 22 '16 at 14:19
  • 1
    $\begingroup$ This may be migrated to "Cross Validated". $\endgroup$ – Gordon Jul 22 '16 at 14:27
  • 1
    $\begingroup$ do you know Kalman filter ? $\endgroup$ – MJ73550 Jul 22 '16 at 16:45

When $X_1$ is unobserved, at iteration $k=1$ of EM, the posterior mean value (when $X_2=3$) is $5.18$ by using an inference algorithm, i.e. Junction tree/Kalman filter. Then the sufficient statistics for $X_1$ is:

$s_1=\Sigma_{i=1}^nx_{i1}=9+4+5.18$ and $s_{11}=\Sigma_{i=1}^nx_{i1}^2=9^2+4^2+(5.18^2+\sigma_{11.2})$

where $\sigma_{11.2}$ is the posterior conditional variance of $X_1$ given $X_2$, when $X_1$ is unobserved.

$s_2=\Sigma_{i=1}^nx_{i2}$ and $s_{22}=\Sigma_{i=1}^nx_{i2}^2$. When $X_2$ is unobserved a term $\sigma_{22.1}$ is also added to $x_{i2}^2$.

So parameters $\mu_{X_1}$ and $\sigma_{X_1}$ are obtained. Similarly, posterior mean of unobserved $X_2$ is also obtained by using i.e. Junction tree inference.

The sufficient statistics for $s_{12}=\Sigma_{i=1}^nx_{i1}x_{i2}$, and $s_{21}=\Sigma_{i=1}^nx_{i2}x_{i1}$.

Mean vector is then: $\mu^{(k)}=(s_1/n, s_2/n)$.

Covariance matrix is: $\Sigma^{(k)}=$$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$=$$\left( \begin{matrix} s_{11}/n-(s_1/n)^2 & s_{21}/n-(s_2/n)(s_1/n) \\ s_{12}/n-(s_1/n)(s_2/n) & s_{22}/n-(s_2/n)^2 \\ \end{matrix} \right)$$ $

Other parameters in the question are obtained. Conditional mean and conditional variance of $X_2|X_1$ is calculated by the formula in the reference of the question. I have validated these formulae with Matlab the result is correct.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.