Let's say I have an arbitrary set of $n$ period returns for $k$ assets, and a given $k \times k$ correlation matrix (of asset returns), which is known a priori.

Does it makes sense, or is it even possible, to think about constructing some kind of measure of whether the set of $n$ returns is consistent with the known correlation matrix (or if they suggest some sort of outlier set)?

Can we rank one set of $n \times k$ returns as being a better match to the given correlation matrix than another set?

Does it make more sense to ask this if one assumes each asset has the same standard deviation of returns?

  • $\begingroup$ what are your independent variables for your matrix? $\endgroup$
    – Nikos
    Dec 28, 2012 at 12:03
  • $\begingroup$ The given individual correlations could, for example, be calculated, from a longer history, from a specific historical period, or from implied volatilities (if available). How do you see this affecting the problem? $\endgroup$
    – Yugmorf
    Jan 2, 2013 at 2:55

4 Answers 4


See "Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size" by Ledoit and Wolf.



One possibility would be to use a similarity based comparison between sets. There are simple geometric distance based measures that work well (euclidean distance is common).

I've seen papers whereby the authors simply run a histogram of all of the correlations and compare to out of sample histograms to determine if the mean of correlations are statistically different or not (can use t-test or ANOVA for instance).

I have been trying to work on generating reliable synthetic multi-asset data (not so easy), so that I can run many such comparison experiments with some kind of confidence intervals.

I would think that comparison measures across different samples and local temporal stability is what ultimately matters.


First, creating a covar matrix from a set of series of unequal lengths is non-trivial and several approaches (eg: one factor or two factor models) will yield different covar matrices. I do see that you said all asset classes have same length (n periods) meaning the above won't apply, but I thought I'd still say this because unless you cherry pick n while working with many assets, most likely you will run into assets with have unequal time series (eg: some company started recently). The point is you need to be consistent about your covar calc technique especially if the covar comes from different sources.

Another warning when fitting returns to covar matrices; I wouldn't advise it since you can have inconsistent results if you use observed returns with a "given" covar matrix for portfolio construction. Returns are observed 'facts' and the covar matrix embodies the relationships, so to avoid inconsistent portfolio results it's best to compute the covar from the raw result info (and always do so with the same technique)

With that out of the way, lets explore measures of "degree of match". Lets define:

  • C = computed covar matrix (from your return series) and
  • C' = given covar matrix that you got from 'somewhere' that you're trying to see if it "fits".

Different measures:

  1. Principle component analysis. Basically determine the eigenvectors and eigenvalues for the COVAR matrix (there will be N of each for an NxN covar matrix). The largest eigenvalues tell you the eigenvectors (or principle components) that capture the strongest 'features' of the COVAR matrix. An introduction to eigenvectors/values. You then compare them in C vs C' to see if the features have changed. If changed, the returns likely do NOT correspond to your 'given COVAR' matrix. Comparison details for C and C':

    • Find the largest eigenvalue of C => pick corresponding eigenvector
    • Find the largest eigenvalue of C' => pick corresponding eigenvector
    • $\theta$ is angle between those two vectors.
    • Absolute value of $\theta$ less than 2-4 degrees => C and C' are similar ("2-4" depends on asset volatility and time period).
    • If "no change" then the returns likely represent the "given" CoVAR (and vice versa)
    • You can tighten the checks by doing the above for not just the largest but also for the smaller components (higher order components) by then checking the 2nd largest, then 3rd largest and so on. If iterate over all N eigenvectors, then you have checked all N dimensions, progressively going from the strongest dimension (at iteration #1) to the weakest dimension (in iteration N).
  2. |C - C'| is a decent choice. Although it's simple, there is no direct physical significance to it because it nets too many individual covars; you are essentially collapsing N dimensions into a single scalar quantity with no prioritization of "strong features" (like above)

  3. Another decent heuristic would be signs and magnitudes of elements within the C-C' matrix. i.e.

    • best = have C-C' as zero i.e. identical covars (duh! :) )
    • ok-to-good = all elements of C-C' are either slightly positive (incl 0) or all slightly negative (incl 0). The actual sign (+/-) is unimportant as compared to the consistency of the signs.
    • bad-to-worst = large absolute values with 1/2 positives and 1/2 negatives

Note that C-C' is a relative measure, so you will need to define your own min-max bounds and then your own intervals for "almost no change". Easiest to do so is look at historical min-max's and adjust them for forward looking (and like every forward looking projection in quant finance, it's art+science). For no change, look at recent times when you recall things being ok in your portfolio and use those values.

  • $\begingroup$ Your answer would be greatly enhanced if you could include information about how you would formulate a confidence interval around your measure. Without this i can't see how it's useful. Thanks. $\endgroup$
    – Yugmorf
    Jan 7, 2013 at 7:53
  • $\begingroup$ @Yugmorf: True, I added that it's a relative measure. I also added a more rigorous measure that you can do if you desire more concrete measures. $\endgroup$ Jan 7, 2013 at 19:04

The short answer is “no.” The long answer is, “it depends on what you intend to do with the ranking.”

First, let us ignore asset classes because you are assuming something that is a priori impossible. We never know the actual population parameters. If we did, we would not need the field of statistics. Nonetheless, we can discuss this question as independent of economics or finance. We can approach this as a statistics question.

You will need to define some things, such as the probability distributions involved and the school of thought, before proceeding. Not all probability distributions have a covariance matrix. In those cases, a correlation may imply something other than what you intend it to mean. You will also need to define a loss function. There may also be other things that may have to be declared, such as the relationships among samples.

Let us begin with a simple case where we have $n$ sets of samples. None of these samples contain overlapping data. Each sample is independent. The parameters are stationary. The set contains variables $X_1\dots{X_k}$. The parameter set contains population parameters $\rho_{i,j}$ where $i$ and $j$ denote separate assets, and $\rho$ is the population parameter of the correlation between $X_i$ and $X_j$. For simplicity, let us also assume that $\rho_{i,j}=\rho_{j,i}$. For some ideas of association, such as those measured by Somers’ D, $\rho_{i,j}\ne\rho_{j,i}$.

We need some form of a loss function. In a decision-theoretic construction, you should choose your own utility function such that $$-U(\rho,\hat{\rho})=L(\rho,\hat{\rho}),$$ where $U$ is your utility function and $L$ is your loss function. You can also adopt a loss function that serves some other theoretical purpose, such as quadratic loss. For our purpose, we could assume quadratic loss. That assumption may not always work, depending on the distributions involved. A careful study should precede your choices in cases where that will not work, such as with many distributions with heavy tails.

Our loss will be defined as $\sum_{\forall{i\ne{j}}}c(\rho_{i,j}-\hat{\rho_{i,j}})^2,c>0.$

For Frequentist methods, Pearson’s rho is the loss minimizing estimator. For Bayesian methods, the posterior mean of the correlation coefficient will minimize the loss. For Likelihoodist methods, the solution is the method of maximum likelihood for most distribution and problems. Implicitly, we have to assume that the probability distributions involved have a finite second moment. In assuming quadratic loss, we have excluded a wide class of real-world probability distributions. One could assume an alternative loss function if that were to happen; however, it would also remove both Pearson’s $\rho$ and the posterior mean as a loss minimizing tool. Some other set of metrics would minimize those alternative loss functions.

Now let us get to your question.

Automatically, if we create ranks $Y_1\dots{Y_n}$, where $Y_q=rank(\sum_{\forall{i\ne{j}}}c(\rho_{i,j}-\hat{\rho_{i,j}})^2),q\in{1\dots{n}}$, then you have your solution.

The problem is that it does not tell you anything with meaning to it. To consider why, imagine the following problem.

You are measuring the height of fifth-grade boys to estimate the population mean. You have randomly selected the Bruno de Finetti School for Boys to make your estimate at. We will eliminate any potential methodological issues away through the use of our magic wand.

Despite our magical powers, a methodological dispute has broken out and has been solved by randomly splitting the students into two groups of equal size using a perfectly fair coin. It is fortunate that we have an even number of students so that no chainsaw was required.

The first group’s sample average was 55 inches tall, and the standard deviation was 2 inches. The second group had a sample average of 55.1 inches and a standard deviation of 1.9 inches. Could we say that our second group had a more precise estimate since the confidence interval was narrower?

No, we definitely could not do so because the treatment was the random toss of a coin, and nothing about random selection can improve precision. Even though you could rank the two groups in terms of narrowness, there is no meaning at all in the ranking due to the very definition of a confidence interval.

The same thing is true for a set of sample correlation coefficients measured against a known set of parameters.

Yes, you can rank them. Their logical ranking is against a utility function. Would there ever be a reason to rank a set? Yes, for demonstration.

If you were teaching the idea of representativeness, you could use the plot of the raw data over the various $1\dots{k}$ variables in the most and least representative sample. If $n$ is large, it might give you an idea of how wide that spread could be.

The advantage in the field of statistics created by measuring an estimated set against a known set is that it permits a statistician to discover, or sometimes simulate, a sampling distribution for $\hat{\rho}$ and from that to know a number of essential properties. Those properties then make it into textbooks as the solution to a known problem so that the researcher can simply plug in the best estimator in a circumstance.

A t-test or an F-test is used because of these properties. Alternative tests are used when those properties do not hold. Likewise, the mean, median, mode, variance, interquartile range, and so on are used because of their properties.

This method is not sensible if you do not actually know the true correlation matrix. It is only a sensible decision if you have worked out the underlying statistical properties. If you assume them into existence and nature does not agree with you, you can create incorrect tools for a real-world problem.

With real-world sets, especially with sets that have missing data or begin at different times, various techniques exist to deal with this problem. Your question would need more definition to help much further.

Your question tag contains the phrase, “modern portfolio theory,” which is a Frequentist construction. Is Modern Portfolio Theory essential to your question because it will also determine your loss function if it is? It will also determine your school of thought and impose boundaries on your possible set of probability distributions. Doing so also restricts how you handle missing data and outliers. Imposing Modern Portfolio Theory onto data imposes many methodological restrictions that are not otherwise present in statistical theory.

For example, Modern Portfolio Theory does not hold in a Bayesian set of assumptions. Indeed, it is wholly foreclosed under most axiomatic constructions because countable additivity is excluded. You cannot create partitions with an infinite number of sets. However, missing data can be solved in a different manner than it can be solved in non-Bayesian methods.

Bayesian methods do not necessarily distinguish between parameters and data. Instead, the discussion tends to be about observables and unobservables. A missing piece of data that was collected but accidentally erased and lost is now unobservable as may be a population mean. On the other hand, a parameter may be known in some cases. It most commonly happens in an engineering setting. In that case, the known parameter is treated in Bayesian methods just like it is observed data because it is known.

Imposing Modern Portfolio Theory also imposes rules on how to handle missing data. It chooses your estimators. It determines your tests.

That is a wonderful thing in some ways. If the theory is strictly correct, then all kinds of decisions have been made for you that you do not have to think about. If it isn’t, then the field of statistics offers a pretty robust set of tools. Still, now you have to think about your real utility function, the actual distributions involved, and possibly what you may need as your set of probability axioms.

If you want to say that 2013 was the year that had observations that were the best compared to a true value for the population parameters, then the answer is “no.” If you want to know the properties of some choice of estimator construction, then a ranking isn’t necessarily the best tool, but it would be if you were demonstrating something.

It is important to remember that tossing a fair coin so that it comes up heads ten times in a row is consistent with a fair coin over a large enough number of samples. If the first sample happens to have ten heads in a row, that is consistent with a fair coin. It is unusual but consistent.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.