One of the most serious shortcomings of covariance/correlation are the assumptions of linearity and normality.

What is the most natural generalization of these measures of dependence when you want to model the dependence structure of extreme events using heavy-tailed distributions, e.g. the Generalized extreme value distribution?

With "most natural generalization" I mean that the classical covariance/correlation is included as a special case when the usual assumptions hold.

(Disclosure: This question was posted at Cross Validated nearly two weeks ago, yet didn't receive any answers)

  • 4
    $\begingroup$ Strictly speaking, there are no assumptions of linearity or normality in the notions of covariance and correlation. The only assumptions needed are: two random variables with finite second moments. $\endgroup$
    – vanguard2k
    Dec 21, 2012 at 10:42
  • $\begingroup$ I tend to agree with vanguard2k. Could you rephrase the question if possible? $\endgroup$
    – Matt Wolf
    Dec 21, 2012 at 11:18
  • $\begingroup$ @vanguard2k, Freddy: Concerning the underlying assumptions have e.g. a look here: symynet.com/fb/quantitative_research_methods/Statistics/… $\endgroup$
    – vonjd
    Dec 21, 2012 at 12:48
  • $\begingroup$ Agree with vanguard2k as well. They may not give you what you want, but they are well defined as long as 2nd moment is finite. $\endgroup$
    – LazyCat
    Dec 21, 2012 at 14:27
  • $\begingroup$ Interesting - Do you have a reference for that. Thank you $\endgroup$
    – vonjd
    Dec 21, 2012 at 15:19

2 Answers 2


Mutual information measures how much knowing one variable reduces uncertainty about another variable. It considers any type of dependency (linear or non-linear), it's measured in bits, and it is widely used in machine learning, computer vision NLP and other fields.


The answer of user27915816 led me into the right direction, yet I think I found an even better generalization:
Distance Correlation (dCor)

There are several reasons for that:

  1. It generalizes classical (i.e. linear) correlation in the sense that linearity is a special case. It gives identical readings for linear dependence.
  2. There are analogs for variance, covariance and standard deviation, so these identities hold: $$\operatorname{dVar}^2_n(X) := \operatorname{dCov}^2_n(X,X)$$ and $$\operatorname{dCor}(X,Y) = \frac{\operatorname{dCov}(X,Y)}{\sqrt{\operatorname{dVar}(X)\,\operatorname{dVar}(Y)}}$$
  3. $dCor=0$ implies true independence, all other readings imply linear or non-linear dependence - Compare the following readings, first linear correlation (source):

enter image description here

and distance correlation (source):

enter image description here

Beware, oversimplification ahead: The reason it shows this behavior is basically that it is the correlation of the characteristic functions of the random variables, i.e. the Fourier transforms of the probability density functions, i.e. a rotation from the time into the frequency domain. Therefore not only linear dependence is being tested but basically all functional dependencies which can be represented by the (periodic) complex exponential function. To get an intuition read also this article: Here.

There are implementations in Excel and R.

  • $\begingroup$ Interesting. Unfortunately the link 'Here' seems to be broken. Could you post it again please? $\endgroup$
    – Yugmorf
    Oct 10, 2013 at 3:26
  • $\begingroup$ The article seems to have disappeared indeed. It is archived here: archive.is/7TVzZ $\endgroup$
    – vonjd
    Oct 10, 2013 at 6:24
  • $\begingroup$ @Yugmorf: Updated the link. $\endgroup$
    – vonjd
    Nov 17, 2020 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.