# Analytical relationship between a covariance matrix and cross-sectional dispersion

Given an expected returns vector and a covariance matrix, one can perform a joint draw and measure the average cross-sectional variation as the standard deviation across returns for a particular joint draw.

Demonstrating the same idea using empirical/historical data, the cross-sectional variation is simply the standard deviation across returns at a point in time. For some intuition, here's a chart plotting cross-sectional dispersion vs. the VIX from a paper by Gorman, Sapra, and Weigand:

Since many shops have a well-designed covariance matrix, rather than looking at the empirical metric to measure dispersion which is noisy and time-varying, I'd rather produce the dispersion metric from an already existing covariance matrix.

What is the analytical relationship between a given covariance matrix and expected returns vector (e.g., a multivariate normal distribution) and the expectation of the cross-sectional dispersion?

-
How do you define the random variable "cross-sectional dispersion"? –  Ryogi Apr 19 '12 at 6:31
Standard deviation of the returns at a point in time. Good question, I updated –  Quant Guy Apr 19 '12 at 13:56
so if r_i and r_j are the returns of each stock, you are looking for the expected value of the product of these two? i.e. E(r_i*r_j) ? –  AdAbsurdum Apr 20 '12 at 15:02
And when you measure standard deviation, are you using the estimator 1/(N-1) * sum(r_i,t * r_i,j) (summed over some time) –  AdAbsurdum Apr 20 '12 at 15:08
Interesting question. I wonder if there really exists a convenient expression for it. I would say you should try playing with the 3-variable version in Mathematica and work to $N$ variables only if you succeed in getting an acceptably simple expression in 3 variables. You may need to switch to variance, and use convecity corrections to adapt that to standard deviation. –  Brian B Apr 20 '12 at 15:47

If $X \sim N(\mu, V)$ is multivariate gaussian, you can write $X = \mu + C Y$ where $Y \sim N(0,1)$ is a standard Gaussian and $C$ is the lower-triangular Choleski matrix of $V$. You can then express $v = \sum_{i=1}^n (X_i - S/n)^2$, where $S = \sum_{i=1}^n X_i$, in terms of $Y$ and $C$.

(I do not reproduce the computations: they are straightforward.) If we just want the expectation, we get: $$E[v] = \sum_i \mu_i^2 - \dfrac1n \sum_{ij} \mu_i \mu_j + \sum_i C_i C_i' - \dfrac1n \sum_{ij} C_i' C_j$$ where $C_i$ is the $i$th row of the Choleski matrix.

This can be simplified: $$E[r] = \text{trace}( \mu \mu' + V ) + \dfrac1n \mathbf{1}' (\mu\mu' + V) \mathbf{1}$$

Here is some R code to check the result. (You may want to divide the result by $n$ or $n-1$, and take the square root of this expectation.)

# Simulations
library(mvtnorm)
f1 <- function(V,mu, R=1000) {
n <- length(mu)
apply( rmvnorm(R, mu, V), 1, function(u) sum((u - mean(u))^2) )
}

# Computations
f2 <- function(V,mu) {
n <- length(mu)
#var(mu)*(n-1) + sum(diag(V)) - sum(V)/n
v <- mu %*% t(mu) + V
sum(diag(v)) - sum(v)/n
}

# Sample data
n <- 10
V <- matrix(rnorm(n*n),n,n)
V <- t(V) %*% V
mu <- rnorm(n)

# Check that the value is the same
f2(V,mu) / mean(f1(V,mu,R=1e5))

-
I can confirm that this works. I still don't really understand it intuitively though... –  John May 4 '12 at 15:02
Vincent's formula can be re-written as var(mu)+trace(sigma)-w'sigmaw, where the w is like the nX1 matrix of 1/ns I mention above. –  John May 4 '12 at 15:07
Well done @VincentZoonekynd –  Quant Guy May 4 '12 at 15:26

For the stationary multivariate normal case, the expected returns vector does not matter. This is because the cross-sectional mean is subtracted out before calculating the standard deviation. The cross-sectional mean can be more conveniently thought of as like the return on an equally weighted portfolio.

Similarly, I would argue that the expected cross-sectional standard deviation will equal the standard deviation of an equally weighted portfolio. In order to calculate the actual cross-sectional standard deviation, you can basically do the same thing and assume an equally weighted portfolio, so they should be analytically the same thing. I ran some tests for five variables and simulated 10,000 times and the numbers came close. They were not perfect, but I suspect that if I took the number of variables and simulations to infinity, then it would work.

-
This is not correct because the cross-sectional dispersion is also a function of the covariances among the securities. I appreciate the work though -- you seem to have identified one of the factors. –  Quant Guy May 2 '12 at 14:29
That's what I'm saying. You take w'sigmaw, where w is an NX1 vector of ones divided by N and sigma is the covariance matrix. As N goes to infinity, the expected cross-sectional variance will go to that for a stationary multivariate normal distribution. –  John May 2 '12 at 17:04
I see. Do you have a MATLAB or R script you can blockquote so I can validate? Thanks again for your efforts –  Quant Guy May 2 '12 at 17:46
It still doesn't come out exactly, even when I increase N quite a bit. So it still could be that there's something else going on. Anyway, here's some code (sorry bout the formatting...): n=250; n_sim=100000; sigma_idio=normrnd(0.1,0.025,[n,1]); sigma_idio=diag(sigma_idio.^2); beta=normrnd(1,0.1,[n,1]); sigma_f=0.1^2*betabeta'; sigma=sigma_f+sigma_idio; w=ones(n,1)/n; mu=normrnd(0,0.025,[n,1]); X=mvnrnd(mu,sigma,n_sim); X_mu=sum(X,2)/n; X_diff=X-repmat(X_mu,[1,n]); X_var=sum(X_diff.^2,2)/n; X_std=X_var.^0.5; mean(X_std) (w'*sigmaw).^0.5 –  John May 2 '12 at 19:04