0
$\begingroup$

I am trying to get some insights on this specific sort of problem from experienced people, as I do not have much experience in this field.

I have a family of features that for simplicity I will just denote it by $S_{i-1}$ (we can think of it as a single feature.). Here $i-1$ means that this feature/signal corresponds to the time interval $[t+(i-1)d, t+id]$. Here $t$ is a fixed starting time and $d$ is the length of the each interval that our data gets updated. Note that $S_{i-1}$ becomes available at the end of the aforementioned interval i.e. at time $t+id$.

Let's denote the average price of an asset over the interval $[t+(i-1)d, t+id]$ by $P_{i-1}$. I have built an ML model that given the signals $S_{i-1}$ can predict $ln(P_i/P_{i-1})$. Note that the actual errors of the predictor can be high but the model predicts the direction (the sign of the return) correctly with a high accuracy (let's say $70$ percent).

Note that this prediction is not tradable, because $S_{i-1}$ becomes available at the end of the interval at a time that is too late to take positions (we should have started to take positions, short or long during $[t+(i-1)d, t+id]$ ).

Note that the same model is capable of predicting the direction of $ln(P_{i+1}/P_{i-1})$ with a very high accuracy as well (this is not tradable as well) but when trying to predict $ln(P_{i+1}/P_{i})$ everything breaks down and model completely loses its capability. Note that this prediction if it had worked would have been tradable. We could have started to take positions in the interval $[t+id, t+(i+1)d]$ and potentially get out of the position in the interval $[t+(i+1)d, t+(i+2)d]$.

So my question is, are these kinds of signals common when trying to predict the returns? (or am I fooling myself to think there is something special happening here) My hypothesis was that if I can manage to get high resolution data and work in higher frequency it might start to work but getting higher frequency data won't be easy, so my intention was to see whether these types of signals have tendency to work in higher frequencies or not. My current frequency is 1 hour.

Improve this question
$\endgroup$
5
  • $\begingroup$ I have no experience in trade direction but is it correct to think about percent win rates when you do direction ? I say this only because if you guessed, you'd tend to be correct 50 percent of the time. So, maybe the percent win rate metric has to be viewed differently ? I'm just saying this to throw it out there. The percent win rate in non-directional is different because it depends on whether you made more on the trade than your transaction cost. To me, that seems like a tougher criteria than direction. $\endgroup$
    – mark leeds
    Commented Aug 6 at 2:27
  • $\begingroup$ @markleeds yes, the accuracy is the percentage of correct directional predictions and also looking at the truth table that includes 4 values: 1) rate of correct directional guesses when price goes up 2) rate of correct directional guesses when price goes down 3) rate of correct guesses when we predict that price will go up 4) rate of correct guesses when we predict that price will go down. All 4 numbers are high and close to 70%. I also calculate the expected returns including the highest possible transaction fee (taker fee) and it produces ridiculously high returns if we could trade it. $\endgroup$
    – user127776
    Commented Aug 6 at 2:38
  • $\begingroup$ I suspect that there's some time overlap between your label and features, since you are using average price to calculate returns. You did not mention how you calculated the average price, there could be information leak if the lookback window is long and covers the time your features are calculated. $\endgroup$
    – autoencoder
    Commented Aug 7 at 1:35
  • $\begingroup$ Also, I find it clearer to shift your feature by one timestamp and use ln(Pi+1/Pi) as label: ie. use S(i-1) to predict ln(Pi+1/Pi). If you have no leak in the label and the result is still bad, then it might be evidence that your feature set is not very predicative. $\endgroup$
    – autoencoder
    Commented Aug 7 at 1:39
  • $\begingroup$ @autoencoder applied the shift hopefully it is more readable. $P_i$ is constructed by 15 min candles. There are 4 such candles in 1 hour, each candles has 4 numbers associated with it (open, close, high , low) I just take the average of the 16 numbers. $\endgroup$
    – user127776
    Commented Aug 7 at 1:51

1 Answer 1

Reset to default
0
$\begingroup$

Posting as an answer here for better readability.

Given what you mentioned in the comments, here's a simple example of your context:

  • K denotes a 15min candle

  • P denotes the price for return calculation, P is the average of OHLC of four candles

  • S denotes the feature set, which might also involve previous candles

As time goes by, one gets the following:

t1: {K11, K12, K13, K14}, S1, P1;

t2: {K21, K22, K23, K24}, S2, P2;

t3: {K31, K32, K33, K34}, S3, P3; ...

At time t1, we gathered market data and calculated feature S1, and we can get a prediction for label ln(P2/P1) using S1. Now OP said the signal is not tradable, but I would argue that it is. However we won't be able to trade the asset at price P1, or the open price of K21, but we can often assume that we could trade at the VWAP of K21, or a VWAP of the next several candles. We will have slippages but we'll be able to trade.

Now, the other problem OP has is that, prediction for ln(P2/P1) is good, prediction for ln(P3/P1) is good, but the prediction for ln(P3/P2) is terrible. The problem is that, the P1 part in label ln(Pn/P1), is calculated using too much past information. Note that P1 is the mean of four OHLCs in the past hour, and although OP didn't mention his/her feature set, I would guess S1 are also features derived from past price/volume data. So there could be large overlap/info leak in the label. A simple adjustment would be to use close to close return as labels, or, as the OP suggests, use ln(P3/P2) as the label.

And we come to the last question: prediction for ln(P3/P2) is bad, which in my opinion is normal. One hour prediction horizon is acutally pretty challenging, if ones tries to do it using just past candle data. Although I have no clue what market/asset class OP is trading at, the market should be decently efficient nowadays. Adding higher frequency data would help, but it again requires more cost to process it.

Improve this answer
$\endgroup$
4
  • $\begingroup$ The asset is crypto and features are not past candle data, they do not include price data but a lot of them have a nature of volume, although not volume in the classical sense like volume of trades in an exchange. The idea is to do some form of "statistical front-running" based on on-chain data (pure front running is very difficult and also competitive). $\endgroup$
    – user127776
    Commented Aug 7 at 7:54
  • $\begingroup$ In this case, how confident are you with your timestamping? Do you collect both the features and the labels in real time on the same machine, if not can you strictly ensure there's no lookhead in the data? Anyway, I would suggest that you change to another label to test. $\endgroup$
    – autoencoder
    Commented Aug 8 at 1:28
  • $\begingroup$ Since you have candles, you could actually train more models on increasing horizons, then you check how the model's predicability decays when the horizon increases. Check if the decay is smooth and if there's a sharp drop at which horizon. For example, if ln(P3/P2) is bad, what about ln(P3/P1.5), ln(P3/P1.25), ..., but here I would use close to close returns. $\endgroup$
    – autoencoder
    Commented Aug 8 at 1:33
  • $\begingroup$ My SQL queries are correct but the data is indexed by another company which I expect it to be correct as well. I did a sanity check and it passed it. Note that S_1 is generated from that specific interval mentioned (does not include past data). S_1 can predict P2/P1 and P1/P0 really well (the latter even better) but P0/P(-1) and P3/P2 become bad, suggesting that probably there isn't a shift in the timestamp. I have tried your idea, while the prediction accuracy increases by shrinking the first interval closer to the current time but it does not become profitable. $\endgroup$
    – user127776
    Commented Aug 8 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.