4
$\begingroup$

In the context of a backtesting engine, is it better to have strategy generate trade signals in the range from -1 to 1 or as exact predicted returns (e.g. -12% or 26%).

The difference lies in how to regress the underlying model: whether to put (the response variable) returns as 1 (for positive) or -1 (for negative) (and use a logit model) or put exact historical return values (and use a linear model).

The reason for asking this is that I found a model that gives me an R^2 of almost 0 (which would seem rubbish in terms of predicting returns), but when backtesting it performs well and is actually a good proxy for relative signal strength (although the returns it gives are 0.17% or -0.09% etc).

It seems that by assigning 1 for positive returns I am losing some of the information; on the other hand trying to predict exact returns seems like a tall order -- I do not like that R^2 might not correspond to backtest results at all.

Which approach is better? Is there some standard literature on this (be it backtesting component design or alpha strategy design)?

$\endgroup$
  • $\begingroup$ How low is R^2? It is common to have pretty low R^2 in this business (unfortunately, otherwise we would all be multi-billionaires). But conversely a high R^2 is not needed to have some degree of profitability. If you are a beginner you may have to reset your expectations of what R^2 are possible.... $\endgroup$ – Alex C Jun 5 '16 at 11:30
  • $\begingroup$ @Alex C if I use exact return value R^2 = 0.0005, if use use +-1 (and a linear, non logit model) R^2 = 0.0008. I am OK with R^2 = [0.1-0.3] but this is basically zero; however the backtest works well. $\endgroup$ – A.L. Verminburger Jun 5 '16 at 12:15
  • $\begingroup$ The current model is based on a paper which seems to make sense. I had instances if the past where I throw in a bunch of predictors, get good cross-validated R^2, but the backtest does not show good results. So far I have always tried to predict exact returns as opposed to the up and down movement; I am now questioning this. $\endgroup$ – A.L. Verminburger Jun 5 '16 at 12:21
  • 1
    $\begingroup$ Interesting. I often use [-1,1] but I don't know a clear theoretical justification. I look forward to answers from other contributors. $\endgroup$ – Alex C Jun 5 '16 at 13:38
  • $\begingroup$ @Alex C Do you use logit models then; and interpret signals as probabilities of up/down price movement? $\endgroup$ – A.L. Verminburger Jun 5 '16 at 13:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.