2
$\begingroup$

I am testing various technical trading rules (TTR) on the cryptocurrency market.

I have already setup some significance tests, to compare the returns and volatilities.

I would now like to test it from a different angle using the ROC model, often applied when backtesting PD [Probability of Default] models. I am just not sure if this is actually feasible in the environment I am trying to apply it.

The logic I have applied until now is that if the output of my TTR is a "BUY" and the underlyings returns that day is bigger than zero, it is a TRUE BUY on the other hand if the return of the underlying is smaller than zero that day I get a FALSE BUY. Having this information I think I should be able to set-up a ROC model.

My question here would be, is this approach actually feasible? Has someone already taken this approach?

$\endgroup$
1
  • $\begingroup$ If the outcomes of the TTR are highly asymmetric (for ex. many small wins, a few big losses) then the high percentage of true buys may give a misleading indication of success, compared to the dollar profit. (Imagine for ex. a TTR of the form: buy at random times, with a 100 USD profit target and a 1000 USD stop loss...). $\endgroup$
    – Alex C
    Jul 28, 2018 at 18:23

1 Answer 1

1
$\begingroup$

So ROC performance metrics are common place in machine learning applications, particularly for classification tasks.

Looking at the way you have setup the question here you wont be able to plot a ROC curve as it relies on a continues output (not discrete). You would however be able to plot a confusion matrix and look at performance metrics such as precision, recall, and F1-score.

All you would need is the actual forecast vs your predicted forecasts.

I did this in python using Sklearn on a simple trend following strategy:

enter image description here

Note: This strategy I built here uses a secondary model to determine when to place a trade based on the primary technical model and that is why all of predictions are labeled as 1. It does however still illustrate the point.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy