# Most natural generalization of covariance/correlation to model dependence of extreme events

One of the most serious shortcomings of covariance/correlation are the assumptions of linearity and normality.

What is the most natural generalization of these measures of dependence when you want to model the dependence structure of extreme events using heavy-tailed distributions, e.g. the Generalized extreme value distribution?

With "most natural generalization" I mean that the classical covariance/correlation is included as a special case when the usual assumptions hold.

(Disclosure: This question was posted at Cross Validated nearly two weeks ago, yet didn't receive any answers)

-
Strictly speaking, there are no assumptions of linearity or normality in the notions of covariance and correlation. The only assumptions needed are: two random variables with finite second moments. –  vanguard2k Dec 21 '12 at 10:42
I tend to agree with vanguard2k. Could you rephrase the question if possible? –  Matt Wolf Dec 21 '12 at 11:18
@vanguard2k, Freddy: Concerning the underlying assumptions have e.g. a look here: symynet.com/fb/quantitative_research_methods/Statistics/… –  vonjd Dec 21 '12 at 12:48
Agree with vanguard2k as well. They may not give you what you want, but they are well defined as long as 2nd moment is finite. –  LazyCat Dec 21 '12 at 14:27
Interesting - Do you have a reference for that. Thank you –  vonjd Dec 21 '12 at 15:19

Mutual information measures how much knowing one variable reduces uncertainty about another variable. It considers any type of dependency (linear or non-linear), it's measured in bits, and it is widely used in machine learning, computer vision NLP and other fields.

-

The answer of user27915816 led me into the right direction, yet I think I found an even better generalization:
Distance Correlation (dCor)

There are several reasons for that:

1. It generalizes classical (i.e. linear) correlation in the sense that linearity is a special case. It gives identical readings for linear dependence.
2. There are analogs for variance, covariance and standard deviation, so these identities hold: $$\operatorname{dVar}^2_n(X) := \operatorname{dCov}^2_n(X,X)$$ and $$\operatorname{dCor}(X,Y) = \frac{\operatorname{dCov}(X,Y)}{\sqrt{\operatorname{dVar}(X)\,\operatorname{dVar}(Y)}}$$
3. $dCor=0$ implies true independence, all other readings imply linear or non-linear dependence - Compare the following readings, first linear correlation (source):

and distance correlation (source):

Beware, oversimplification ahead: The reason it shows this behavior is basically that it is the correlation of the characteristic functions of the random variables, i.e. the Fourier transforms of the probability density functions, i.e. a rotation from the time into the frequency domain. Therefore not only linear dependence is being tested but basically all functional dependencies which can be represented by the (periodic) complex exponential function. To get an intuition read also this article: Here.

There are implementations in Excel and R.

-
Interesting. Unfortunately the link 'Here' seems to be broken. Could you post it again please? –  Yugmorf Oct 10 '13 at 3:26
The article seems to have disappeared indeed. It is archived here: archive.is/7TVzZ –  vonjd Oct 10 '13 at 6:24