I am trying to classify similar looking curves of a timeseries and was wondering what is the best algorithm to research. Reading R, it looks like k-means clustering could be applied - but I don't know if there are better algorithms. Any pointers is much appreciated. My concern is if k-means could group points around a neighborhood of specific mean, but what if two curves with similar mean show different variance.
$\begingroup$
$\endgroup$
1
-
$\begingroup$ QuantGuy - I need to think more and thanks for the link. The data is basically a list of pre-conditions (say related to volume/RSI/momentum etc) of a stock and I am trying to cluster these pre-conditions based on what the stock has done after those conditions are met. My approach was to first cluster the resultant curves of the stock first and then based on similar looking resultant curvers, cluster the pre-conditions for prediction. Hope the approach makes some sense. $\endgroup$– fooledbypatternCommented Dec 6, 2011 at 17:41
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
If the means are similar, then K-means will not do a great job. I would generate new features, perhaps based on higher moments of the distribution or some other properties (auto-correlation, summary of spectral density, etc.).
Using this new set of features, If you see separation of two curves when you plot draws in feature space then k-means would be an effective grouping algorithm.
It's hard to diagnose the best classification tool without better understanding the data generating process. For example, it might make sense to use a mixture of gaussians model instead.
This R task view page on clustering will provide a broad list of various tools you can use.
answered Dec 6, 2011 at 2:39
-
$\begingroup$ Do you ever throw in regression coefficients, like regression against indices (including sector/industries)? $\endgroup$– JohnCommented Dec 20, 2013 at 22:19
$\begingroup$
$\endgroup$
3
We have used Neural Networks for this purpose. What we did was this: select a set of characteristics which can be directly calculated from the time series (e.g., mean, standard deviation, skewness, kurtosis, p-value from a normal fit, ACF stats etc.), and then run an NN to learn from a set of timeseries about how to classify them. We were trying to classify which among a set of models would produce the best forecasts for a series. We had the forecast answers, since we used the M-Competition data.
Unfortunately, the results weren't great. May be we were looking at wrong set of characteristics, may be our classification were wrong. But this is definitely an way to solve the problem you mention. I would be interested in knowing if you get any success in this work.
I would also be interested if someone wants to collaborate on this investigation. Please see my profile for contact information.
-
1$\begingroup$ Why do you think the results were not satisfactory? I would suggest that the characteristics of the first series did not match that of the set. Did you gain any additional insight into what was mentioned? $\endgroup$ Commented Dec 6, 2011 at 21:44
-
$\begingroup$ @strimp099: The results were not satisfactory because, after the training was done, we tried the NN on a test set. The success rate was only ~20%, which was close to randomly selecting one of the method as the best method. This could mean a variety of things: (1) We could have missed some important characteristic, (2) The construction of NN was faulty, (3) The training data was too noisy etc. We did not really gain much additional insight from the exercise, but I still think this is a very viable route. $\endgroup$– Samik RCommented Dec 13, 2011 at 21:07
-
$\begingroup$ Samik: Update: Even for me, I did not find kmeans to give satisfactory results. One thing that I did not realize earlier was the fact that the order of data points is not being considered in kmeans. I need to consider a variant of kmeans that consider sequence. Doesn't look like R has any such implementation. $\endgroup$ Commented Jan 2, 2012 at 0:11
$\begingroup$
$\endgroup$
My concern is if k-means (AKA lloyd algorithm) could group points around a neighborhood of specific mean, but what if two curves with similar mean show different variance.
K-means will groups curves which are close to an "average curve", but not necessarily close to a constant mean. Three very separated clusters can have the same mean. One being a cluster whose average is oscillating, the other one around an affine function, and the third one a constant function. In a sense you need to consider that each curve is a n-dimensional vector in the case where n is the number of points of discretization.
If you really don't want to use K-means, there is always the so-called "CLVQ algorithm" (competitive learning vector quantization) It should give similarresults.
More generally, quantization / clustering has extended to the functional case in "Functional quantization of Gaussian processes" by Luschgy and Pagès (Journal of functional analysis). The main result is that if you assume that your curves are independent draws of a Gaussian process, the optimal quantizer of that Gaussian process will span a principal space of its covariance operator, in other words, it will span the same finite-dimensional space as the first Karhunen-Loève eigenfunctions. This result as then been used by Tarpey and Kinateder in "Clustering Functional Data" (Journal of classification).