3
$\begingroup$

This question is sort of a continuation of this, but i wanted to share the progress i made and ask for help on the part where i'm stuck.

The short story is that i have a pattern stored in a simple array of data, then i have a dataset and i need to check for occurrences of the pattern i specified on the other dataset.

Here is what i did:

  1. Get a set of OHLC data on a pandas dataframe
  2. Compute local minima and maxima for that OHLC data
  3. Get an array of local minima and maxima
  4. Normalize the array of minima and maxima by converting it to an array of number, where every number is the variation from the previous point of local minima/maxima.

In terms of code, this is how you can find local minima and maxima on a range:

df['min'] = df.iloc[argrelextrema(df.Open.values, np.less_equal, order=n)[0]]['Open']
df['max'] = df.iloc[argrelextrema(df.Open.values, np.greater_equal, order=n)[0]]['Open']

Dataframe:

         Open       min       max                Date
Loc
0    0.000336  0.000000  0.000336 2020-07-06 12:00:00
6    0.000330  0.000000  0.000330 2020-07-06 18:00:00
12   0.000320  0.000320  0.000000 2020-07-07 00:00:00
15   0.000328  0.000000  0.000328 2020-07-07 03:00:00
18   0.000320  0.000320  0.000000 2020-07-07 06:00:00
27   0.000330  0.000330  0.000000 2020-07-07 15:00:00
32   0.000351  0.000000  0.000351 2020-07-07 20:00:00
34   0.000342  0.000342  0.000000 2020-07-07 22:00:00
42   0.000368  0.000000  0.000368 2020-07-08 06:00:00
48   0.000381  0.000000  0.000381 2020-07-08 12:00:00
54   0.000361  0.000361  0.000000 2020-07-08 18:00:00
55   0.000361  0.000361  0.000000 2020-07-08 19:00:00
61   0.000378  0.000000  0.000378 2020-07-09 01:00:00
65   0.000367  0.000367  0.000000 2020-07-09 05:00:00
69   0.000375  0.000000  0.000375 2020-07-09 09:00:00
72   0.000373  0.000373  0.000000 2020-07-09 12:00:00
75   0.000388  0.000000  0.000388 2020-07-09 15:00:00
78   0.000378  0.000378  0.000000 2020-07-09 18:00:00
86   0.000411  0.000000  0.000411 2020-07-10 02:00:00
90   0.000395  0.000395  0.000000 2020-07-10 06:00:00
92   0.000402  0.000000  0.000402 2020-07-10 08:00:00
96   0.000417  0.000000  0.000417 2020-07-10 12:00:00
99   0.000411  0.000411  0.000000 2020-07-10 15:00:00
105  0.000433  0.000000  0.000433 2020-07-10 21:00:00
108  0.000427  0.000427  0.000000 2020-07-11 00:00:00
116  0.000479  0.000000  0.000479 2020-07-11 08:00:00
118  0.000458  0.000458  0.000000 2020-07-11 10:00:00
123  0.000467  0.000000  0.000467 2020-07-11 15:00:00
133  0.000425  0.000425  0.000000 2020-07-12 01:00:00
137  0.000447  0.000000  0.000447 2020-07-12 05:00:00
141  0.000434  0.000434  0.000000 2020-07-12 09:00:00
145  0.000446  0.000000  0.000446 2020-07-12 13:00:00
149  0.000434  0.000434  0.000000 2020-07-12 17:00:00

Then convert this dataframe in a simple list of Minima and Maxima: [0.0003361, 0.0003296, 0.0003197, 0.0003278, 0.0003204, 0.0003301, 0.0003513, 0.000342, 0.000368, 0.0003809, 0.0003611, 0.0003781, 0.000367, 0.0003747, 0.0003727, 0.0003884, 0.0003783, 0.0004105, 0.000395, 0.0004022, 0.0004168, 0.0004107, 0.0004334, 0.000427, 0.0004793, 0.000458, 0.0004668, 0.0004245, 0.0004472, 0.0004344, 0.0004457, 0.0004335]

And then convert it again to a simple array of percentages:

[-1.9339482296935422, -3.00364077669902, 2.533625273694082, -2.2574740695546116, 3.027465667915112, 6.4222962738564, -2.647309991460278, 7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172, 4.212503353903944, -2.600411946446969, 8.511763150938416, -3.775883069427527, 1.8227848101265856, 3.6300348085529524, -1.4635316698656395, 5.527148770392016, -1.476695892939546, 12.248243559718961, -4.443980805341117, 1.9213973799126631, -9.061696658097686, 5.347467608951697, -2.8622540250447197, 2.6012891344383067, -2.737267220103202]

From the previous Dataset, i extracted a pattern, which is the following:

Pattern = [7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172]

When charted, it looks like that:

Until now, everything works. Now i need to find the pattern in that figure in other datasets. That pattern is made of the following values: Pattern = [7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172]

So i will need a way to detect the previous pattern on another dataset. For example, if the other target dataset will be that:

[-1.9339482296935422, -3.00364077669902, 2.533625273694082, -2.2574740695546116, 3.027465667915112, 6.4222962738564, -2.647309991460278, 7.602339181286544, 3.5054347826086927, -5.198214754528746, 4.7078371642204315, -2.9357312880190425, 2.098092643051778, -0.5337603416066172, 4.212503353903944, -2.600411946446969, 8.511763150938416, -3.775883069427527, 1.8227848101265856, 3.6300348085529524, -1.4635316698656395, 5.527148770392016, -1.476695892939546, 12.248243559718961, -4.443980805341117, 1.9213973799126631, -9.061696658097686, 5.347467608951697, -2.8622540250447197, 2.6012891344383067, -2.737267220103202]

How do i find the parts of this dataset that will be most similar to the pattern i defined by myself?

Possible solutions that i don't know how to use: I've been suggested to use stumpy or Python-DTW (Dynamic Time Warping). But for both there aren't examples on this particular matter, so if anyone can help me out on this, it would be a lot appreciated. Any kind of advice, library, example, article on how to solve this problem is appreciated. I've been dealing for a lot on this problem and i feel like i'm only missing the final part to finally solve it

TL;DR I'm trying to find patterns specified by myself on OHLC datasets. To do that, i converted the OHLC data to a set of local minima and maxima. Now i need to understand how to compare a specific pattern to a target dataset and detect where the dataset is most similar to the pattern i specified.

Improve this question
$\endgroup$
1
  • $\begingroup$ look into trend scanning $\endgroup$
    – develarist
    Jul 13, 2020 at 3:59

1 Answer 1

Reset to default
2
$\begingroup$

your problem is pattern recognition. considering you already identified the desired output pattern (entry/exit points), you can use supervised methods of machine learning to train.

many are available, a support vector machine for instance, recommend you to check the scikit learn module out, it has practical and fast implementations.

you would have to divide your sample between training and testing, possibly increasing its effectiveness with cross validation methods, e.g. k-fold

also you might want to note that the order=n in the argrelextrema function, introduces a parameter which makes the learning process vulnerable to overfitting.

to avoid this, and considering the volatility of your time series might be variable (optimal n might vary), you could consider using hidden markov models for regime changes identification.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you really a lot for your answer! This is a lot interesting, i'm now looking for example on this matter. The problem with the variable n is that it allows me to specify the number of intervals on which the local minima and maxima need to be found; using hidden markov models is still a bit out of my knowledge, but it's definitely something i should do to improve the system, in the future $\endgroup$
    – Jack022
    Jul 12, 2020 at 20:45
  • $\begingroup$ thanks. the scikit learn package documentation linked in the reply is full of examples, have a read $\endgroup$
    – John
    Jul 13, 2020 at 1:16
  • 1
    $\begingroup$ do not use k-fold cross-validation for financial time series. Instead there is combinatorial purged CV $\endgroup$
    – develarist
    Jul 13, 2020 at 4:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.