17
$\begingroup$

While I spend most of my StackExchange time on MathematicaSE, I'm in the business and follow the questions and answers on this site with great interest.

Recently questions like the following (and some others):

And even something like this from the Wolfram blog:

have me thinking about the different approaches available to analyze time-series similarity.

These approaches could include:

  • covariance,
  • correlation,
  • co-integration,
  • PCA,
  • factor analysis,
  • entropy,
  • graph theory,
  • cluster analysis, and
  • probably others

They certainly have different uses and apply to different aspects of time-series. They all seem to have something to contribute to understanding time-series similarity.

So this leads me to a number of questions (or maybe just restatements of the same question):

  • Can anyone provide a good intuition describing how they all relate to one another?

I hope here for something beyond things like some of the excellent discussion of the differences between correlation and co-integration and the kinds of series to which they apply that have appeared elsewhere on this site. I hope for something more along the line of...

  • Does some point of view or insight exist that provides a better idea of all of these measures?

  • From some meta-perspective could one view these approaches as aspects of some broader idea?

  • Could some meta-perspective recast all of these approaches so one could view them in similar units of measure?

Not even certain this is possible. Even as I write out this post it seems like asking someone to deliver something like a unified field theory of time-series similarity.

Still, if possible it might prove useful so, maybe the questions will spark an interesting answer or two.

Of course, any recommendations of papers or other resources that explore any of this appreciated.

...

Updated 15 Aug 2012 2:00 PM EDT

The following paper gives an example of the kind of thinking that moves in the direction of a broader answer to this question: Empirical Entropy Manipulation and Analysis.

The section on casting (at least some) PCA problems as entropy maximization problems follows:

enter image description here enter image description here

And yes PCA can be done by eigenvalue decomposition of a data covariance (or correlation) matrix.

So, clearly some of these kinds of analysis have some kind of relationship.

Perhaps some view can include more of them. Maybe an entropy or information description of correlation or cointegration. I'm not sure, but to me it seems a interesting question.

Improve this question
$\endgroup$
3
  • 1
    $\begingroup$ Would you like to make the question more specific? E.g. what is the relation between PCA and the following method in information theory? - otherwise, as you rightly notice, it looks like a collection of buzz words and the quest for unified theory of everything. $\endgroup$
    – LazyCat
    Aug 15, 2012 at 17:38
  • $\begingroup$ @LazyCat -- Not certain what you mean or imply by "buzz words". Everything that I mention from correlation to entropy represent meaningful and frequently used methods of analysis. I hoped someone might know of some work from somewhere that began to think of these things as parts of something larger. Both science and mathematics often advance from someone recognizing or establishing such a differential perception. It would seem a useful thing if achievable. Neither does anything immediately come to mind that would preclude it. I feel quite comfortable letting the question stand. $\endgroup$
    – Jagra
    Aug 15, 2012 at 21:36
  • $\begingroup$ I mean that if we go nitpicking, Graph Theory is not a method of analysis of time series. More seriously, I think a question about the relation between two concrete methods of analysis, similar to your example fits the board much better, than a demand for survey of all methods and their interrelations. $\endgroup$
    – LazyCat
    Aug 16, 2012 at 13:24

1 Answer 1

Reset to default
11
$\begingroup$

Here is a structured list of your bullet points:

  • covariance,
  • correlation,
  • PCA,
  • factor analysis,

Are similar. They are based on Gaussian assumptions (i.e. correlations means dependencies) and try to identify common factors (i.e. a variable in small dimension) explaining the observed relationships.

  • co-integration

is more specific in the sense that you focus on capturing relationships which residuals are stationnary (say i.i.d. to make it simpler).

  • entropy,

is the non linear version of the first list. The only bad point is that when a mutual information (it is the correct term) analysis is positive, you have no guarantee that you will be able to build the adequate model to capture the identified relationship. Entropy gives you the existence of a non linear model, but no clue to find it.

Moreover, there is a possible mix between entropy and PCA: the ICA (Independent Component Analysis). It focuses on finding not few orthogonal factors but few independent factors.

  • graph theory,
  • cluster analysis,

are more adequate for discrete relationships: they will given you homogenous groups of stocks with respect to binary variables (increasing vs decreasing for instance and not take into account the intensity of the vatiations). It is a good complement to a factor or PCA analysis, to help you to understand the meaning of the factors.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Almost a year later and I find I keep coming back to your answer. Thanks again. I wish I could vote it up more. $\endgroup$
    – Jagra
    Jul 12, 2013 at 15:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.