# Calculate Exponentially-Weighted Covariance Matrix over Finite Window

I have an (n,m) array (specifically containing asset returns over n days for m assets). I'm trying to calculate the rolling exponentially-weighted covariance matrix for these assets over this time frame, but I want to limit how much data with which each covariance matrix is calculated.

To be more specific, I'm wanting to calculate these covariance matrices using 20-observation half-lives, but I don't want to include more than 40 observations in each of these calculations.

I've come as far as constructing a pandas DataFrame which has a shape of (n, m, 40), so each value of n contains the last 40 observations of the m assets. I was thinking I'd be able to calculate a single exponentially-weighted covariance matrix with 20-observation half-life at each n using the data in that row , but I'm coming up short. Am I able to calculate it this way or is there a different approach I should take?

Edit: I'm looking to avoid for loops in this solution.

• It's not clear ( atleast to me ) what 20 observation half lives means ? There is a parameter in exponential smoothing usually denoted as $\lambda$ and this parameter maps to some half-life by solving $\lambda^{HL} = \frac{1}{2}$ where $HL$ is the half life you want. May 17, 2022 at 4:49
• Th above assumes that the ES model is: $\hat{Cov}_{t} = \lambda \times \hat{Cov}_{t-1} + (1-\lambda) \times Cov_{t}$. May 17, 2022 at 4:52
• I assume the OP means lambda^20 = 0.5 for the comment re half life. May 17, 2022 at 5:05
• HI: I don't know if its extremely fast but maybe find the function, CovEWMA, that he refers to on page 3 of this. faculty.washington.edu/ezivot/econ589/… May 17, 2022 at 14:02
• Note that in the above, Eric's $(1-\lambda)$ is my $\lambda$. Things can get messy ( and wrong ) quickly if one is not careful about how $\lambda$ is defined so be careful with that. May 17, 2022 at 14:05

If I understand you correctly, you are aiming to compute a series of covariance matrices based on windows of your return data. To this end, let $$X$$ denote the $$n\times k$$ matrix of observed returns for $$n$$ dates and $$k$$ instruments. Further, you have a window of length $$h.

Then, the typical entry $$(k,l)$$of each component covariance matrix is calculated as (assuming zero mean)

$$C_i(k,l)=\sum_{t=i}^{h+i-1}w_tx_{k,t}x_{l,t}=X_i^TWX_i$$

where $$X_i$$ is the $$i$$th window of the data matrix, i.e.

$$X_i\equiv \begin{pmatrix} x_{1,i}&x_{2,i}&\ldots&x_{k,i}\\ x_{1,i+1}&x_{2,i+1}&\ldots&x_{k,i+1}\\ \ldots&\ldots&\ldots&\ldots\\ x_{1,i+h-1}&x_{2,i+h-1}&\ldots&x_{k,i+h-1} \end{pmatrix}$$

and $$W$$ is a diagonal matrix of the weights. The corresponding computation is a simple for-loop. HTH?

• Thanks for the answer @Kermittfrog. You are understanding me correctly, and your answer makes sense to me. However, and I neglected to state this in the initial question, I'm hoping to find a vectorized way of calculating this (I've updated my question to include this now). Being that I have an (n,k,h) DF, I was hoping I could calculate the matrices using pd.DataFrame.ewm().cov() along a specific axis, but I haven't been able to make that work. May 17, 2022 at 13:20
• Vectorisation only makes sense if 1) your data is arranged neatly in memory (the prog. language should help), 2) the operator is optimised accordingly (again, your prog. language will do that). What you are looking for is an operation that maps an $n\times k$ matrix into a $n\times k \times h$ tensor. You can do that, of course, using tensor algebra. There should be packages available ('tensor' in both R and Python). But note that this would require both $X$ and $W$ to be inflated in such a way that their tensor products will yield the $n\times k\times h$ object... May 18, 2022 at 6:58
• Long story short: If there's no yet-predefined-function available, simply use a loop. Unless you require low latencies, but then you'd not be using Python to begin with, no? May 18, 2022 at 6:59
• Fair enough! Python is the sole language I'm working with right now, and I'm trying to be as efficient with it as I can. I might look into creating a Cython file if the for loop is the only way I can make this happen. Thanks for the feedback. May 18, 2022 at 13:20