EM for conditional Gaussian model

Question

Let $$X_1\sim N(\mu_{X_1},\sigma_{X_2}^2)$$ $$X_2\sim N(\mu_{X_2}, \sigma_{X_2}^2)$$ where $\mu_{X_2}=c+aX_1$. Also, I have data $D$ (with missing values on $X_1,X_2$).

How can I update/estimate the parameters $\mu_{X_1},\sigma_{X_1},\mu_{X_2},a,c,\sigma_{X_2}$ using EM? i.e. what is the formula for updating $\sigma_{X_2}$?

My model is a conditional Gaussian, which is a conditional form of a bivariate Gaussian $(X_1,X_2)$ with mean vector $(\mu_1,\mu_2)^\top$ and covariance matrix $$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$

Here is a reference to convert bivariate Gaussian to conditional Gaussian: $$\mu_{2|1}=\mu_{2}+\Sigma_{21}\Sigma_{11}^{-1}(X_1-\mu_1)\quad,\quad \Sigma_{22|1}=\Sigma_{22}-\Sigma_{21}\Sigma_{11}^{-1}\Sigma_{12}$$ which yields my model.

It seems that when $X_1$ has different observations and $X_2$ is unobserved, the variance of $X_2$ remains unchanged.So how to update the $\sigma_{x_2}$? Do I need to estimate covariance matrix?

Initial setting for the model $$X_1\sim N(5,7)\quad,\quad X_2\sim N(0.5X_1,8)$$

Data:

$$X_1:\operatorname{9\,\,4\,\,NA}$$

$$\quad X_2:\operatorname{NA\,\,NA\,\,3}$$

Answer 1

When $X_1$ is unobserved, at iteration $k=1$ of EM, the posterior mean value (when $X_2=3$) is $5.18$ by using an inference algorithm, i.e. Junction tree/Kalman filter. Then the sufficient statistics for $X_1$ is:

$s_1=\Sigma_{i=1}^nx_{i1}=9+4+5.18$ and $s_{11}=\Sigma_{i=1}^nx_{i1}^2=9^2+4^2+(5.18^2+\sigma_{11.2})$

where $\sigma_{11.2}$ is the posterior conditional variance of $X_1$ given $X_2$, when $X_1$ is unobserved.

$s_2=\Sigma_{i=1}^nx_{i2}$ and $s_{22}=\Sigma_{i=1}^nx_{i2}^2$. When $X_2$ is unobserved a term $\sigma_{22.1}$ is also added to $x_{i2}^2$.

So parameters $\mu_{X_1}$ and $\sigma_{X_1}$ are obtained. Similarly, posterior mean of unobserved $X_2$ is also obtained by using i.e. Junction tree inference.

The sufficient statistics for $s_{12}=\Sigma_{i=1}^nx_{i1}x_{i2}$, and $s_{21}=\Sigma_{i=1}^nx_{i2}x_{i1}$.

Mean vector is then: $\mu^{(k)}=(s_1/n, s_2/n)$.

Covariance matrix is: $\Sigma^{(k)}=$$\left( \begin{matrix} \Sigma _{11} & \Sigma _{12} \\ \Sigma _{21} & \Sigma _{22} \\ \end{matrix} \right)$$=$$\left( \begin{matrix} s_{11}/n-(s_1/n)^2 & s_{21}/n-(s_2/n)(s_1/n) \\ s_{12}/n-(s_1/n)(s_2/n) & s_{22}/n-(s_2/n)^2 \\ \end{matrix} \right)$$ $

Other parameters in the question are obtained. Conditional mean and conditional variance of $X_2|X_1$ is calculated by the formula in the reference of the question. I have validated these formulae with Matlab the result is correct.