7
$\begingroup$

Say I have found a way through technical analysis to predict how stocks would behave with 58% accuracy, how good is this percentage?

$\endgroup$
3
$\begingroup$

In the long term you will underperform buy & hold because you need an accuracy of at least 65%.

See these papers for more:

$\endgroup$
  • $\begingroup$ what if she is doing HFT? My understanding is that with 58% accuracy forecast, she will retire rich long before your "long term" will expire. $\endgroup$ – LazyCat Jun 29 '15 at 14:07
  • $\begingroup$ @LazyCat: See my question: quant.stackexchange.com/questions/1787/… $\endgroup$ – vonjd Jun 29 '15 at 14:22
  • $\begingroup$ OK, so what? Don't see how it contradicts my scenario. $\endgroup$ – LazyCat Jun 29 '15 at 15:37
  • $\begingroup$ @LazyCat: I didn't want to contradict you but I think you are interpreting too much into this question. Do you really think that the OP is interested in theorizing about HFT when she asks a question about technical analysis ;-) $\endgroup$ – vonjd Jun 29 '15 at 15:42
  • 1
    $\begingroup$ you are right, - I've overlooked the part about technical analysis in the question. $\endgroup$ – LazyCat Jun 29 '15 at 22:17
3
$\begingroup$

It's not bad but you have to backtest the method out-of-sample.

Say you have discovered an indicator that works 100% in history, you still cannot be sure if it works next time.

Another advise is you might want to investigate the distribution of loss when your system fails to work. If your system delivers 1% every time you trade, and loses 10% each time it fail, you probably wind up losing in the end.

$\endgroup$
2
$\begingroup$

Directional forecast is insufficient. You could have a signal that has 100% accuracy and you would not necessarily be able to profit from it because of transaction cost, implementation etc.

$\endgroup$
  • 1
    $\begingroup$ I disagree, if you had a perfect forecast you definitely would outperform buy & hold even after transaction costs etc. $\endgroup$ – vonjd Jun 27 '15 at 9:13
  • 1
    $\begingroup$ @vonjd I don't know where you're getting the time horizon and buy-and-hold from. I could know for sure (100% accuracy) that price will be higher within n time steps and I would still not necessarily profit from it because my execution time is n + epsilon. I can always find some epsilon such that this is trivially true regardless of the time horizon of the 'trend' we are talking about. $\endgroup$ – madilyn Jun 27 '15 at 10:50
2
$\begingroup$

It's not unusual to find a financial time series with positive trend samples biased between 55-60%, depending on the period sampled. Stocks tend to have an upward drift over the long run. When you account for the drift, I would say, that number is really not much better than chance.

A better way to verify your question would be to make certain to build your estimates and models (with parameter optimization) on a certain portion of the sample and comparing out of sample results of your model's performance to the actual performance of the stock series under the same out of sample time frames. Using a rolling window is commonly used.

And also, make sure to look at the actual expectation or performance of the actual returns over the comparison period, as simple directional signs can be close to meaningless without values for context.

Incidentally,this researcher tested 1 million machine learning models using data from 93 - 2008 to predict SPY ETF direction and acheived an astounding 78% in sample performance. When the trained model was tested over the next two years out of sample data, it dropped all the way to 50%... not so remarkable, and a good example of why its really important to consider validating data over different periods.

$\endgroup$
  • $\begingroup$ Although I agree the deeper question is whether an out-of-sample performance of 58% would be good. $\endgroup$ – vonjd Jun 28 '15 at 10:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.