Say I have time series data for $N$ assets, where for the longest existing asset I have data from $t_0=0$ to $T$, but for several other assets I only have data from say $t_0+k$ to $t_0+l$ for some $0<k<l<T$.
Now, if I want to do portfolio optimization with rolling estimation of the covariance matrix using a window of size $m$, what are some good ways to deal with the assets for which I'm missing data points inside the window?