1
$\begingroup$

I know this question will be quickly destroyed and my account summarily banned, but I just have to ask:

For a trader using machine-learning algorithms (SVMs, ANNs, GAs, Decision Trees) for quantitative finance, without seasoned financial intuition, what would be considered a good confidence / success rate?

I know this will depend on the following, as well as other items I'm not aware of:

-Market / Sector (stocks, commodities, FOREX, etc.)
-Principal investment
-Frequency of trades
-Share price
-News Volatility of sector
-Range of dates used for datasets

Please feel free to list other considerations... But in the end, to make the question crystal clear I'd really like a target number. 75%? At 60% I would be roughly taking 1 step forward for every 10 steps taken. Any less and I might as well flip a coin. If it varies, please list the considerations under which they do so. If possible, it would be preferable to use these models to support trades over a period of days rather than seconds/minutes.

If you have other suggestions for how to go about things based on low/high, principal, markets to consider, share prices, etc. please feel free. If my question does not make sense, please tell me why. Thank you.

UPDATE

-At this point I was simply trying to predict up and down movements over a 5 day period. Simple. 45%.
-Free Yahoo data be my market data source... daily quotes. Wasn't sure if intra-day information would be helpful.
-I've attempted ANNs, SVMs, and some GAs so far.
-I wasn't looking for real-time trading, but instead looking to identify regular tides over a several day period.
-Maybe if I can get my error high enough, I can simply trade opposite my predictions! (no, seriously though)

Improve this question
$\endgroup$
3
  • $\begingroup$ If this post belongs on CrossValidated, then I apologize. Please migrate. Otherwise, please explain a downvote. $\endgroup$
    – poorly_built_human
    Jun 23, 2014 at 11:51
  • $\begingroup$ I think this question is definitely appropriate here. I am no expert on this topic, but I think if you use models like that you aim for a success rate that is well above $50\%$ with the amount depending on the cost of implementing the trading signal so that you place bets with an expected value >0. Further more, you would want to add some additional margin (for estimation and modelling error or changing environment for example). It definitely depends on the cost of getting in and out of the position. Thats why there can't be a single target number. $\endgroup$
    – vanguard2k
    Jun 23, 2014 at 12:29
  • $\begingroup$ I flagged this question for closure. I appreciate your honesty about being a beginner but this question seems to be outside the scope of this SE. Firstly, this question is too broad and opinion-based - as the responses suggest - there's no right answer. I would be OK with a 50.1% success rate at a coin flip and a 0.1% success rate at winning a billion dollars. I'd need to know the transaction costs etc. And even so, there's no final answer because a positive expectancy strategy may be within someone's risk appetite but outside another's. $\endgroup$
    – madilyn
    Jun 23, 2014 at 18:50

4 Answers 4

Reset to default
2
$\begingroup$

You're thinking about this the wrong way, in my opinion. Win/loss percentage is worthless in isolation. You must consider the symmetry of your winners and losers. You can have a win % of only 40% and still have a wonderful strategy if your your winners are significantly larger than your losers (this is the classic trend follower PnL distribution).

So, you could flip a coin and see 50% prediction accuracy. That would be outstanding if your winners are 2x larger than your losers.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you very much, you've been the most helpful so far. I realized that I was not taking into account many factors, but didn't know what they all were. Can you elaborate on your answer a little more before I give you the green check? Is there a specific formula (other than standard risk assessment) for assessing individual trades? I imagine there is a fair amount of game theory involved. Also, is it advisable to map out the movements in multiple smaller increments such that if a move doesn't go as predicted the stop-losses can be activated, or simply hedging bets? ...(continued) $\endgroup$
    – poorly_built_human
    Jun 24, 2014 at 11:40
  • $\begingroup$ ...I have read this and this and a few other links on risk assessment, but I'm still having trouble understanding what value machine learning can be if your chances are 50/50. Thank you again. $\endgroup$
    – poorly_built_human
    Jun 24, 2014 at 11:40
2
$\begingroup$

"Success rate", in the sense of winning (W) vs. losing (L) percentage of trades, is almost completely meaningless if taken alone as a trading metric. With a trend-following (TF) trading strategy, where you quickly exit any trades that start to become losers (i.e. cut your losses fast) but let your profits run, a typical win-rate would be around 35% or so, and this is excellent if your average win amount is 3 times your average loss amount. In this case your expected return is 0.35x3 - 0.65x(1) = +0.40 times R, where R is the amount you RISKED per trade. Conversely, with a Counter-Trend (CT) / Mean-Reversion trading strategy, where your winning amount per trade might not be more than about 1.2 times R, so you will need a win rate of at least 65% to be about equally profitable, i.e. 0.65x1.2 - 0.35x1 = +0.43 times R per trade. The above numbers are reasonable "ball-park" figures for good real-life trading systems. In fact if you average 0.4*R per trade with either type of system, you will make a LOT of money and you can certainly consider yourself very successful as a trader. As you can see from the example, "Success" does not necessarily equate to a high win rate at all. The win rate that you NEED for financial success in trading will depend entirely on what is your preferred trading style.

Improve this answer
$\endgroup$
1
$\begingroup$

OK, in for a penny, in for a pound :-)

First, let us assume that you have symmetrical critical levels higher and lower (call them “target” and “stop” if you will). Only in this case is the hit rate relevant.

Even, then the hit rate is a function of time. If you take a 5d/1w view, then being 51% right is very different to being to being 51% right on a 21d/1m, which is very different to 51% right on a 65d/3m view. All of these annualise to very different annual hit rates. Hit rates “root time”, like eg volatility. [strictly speaking, they probit-time, but who cares :-)]

The punchline is that lower hit rates at higher frequencies are equivalent to higher hit rates at lower frequencies.

Theoretically (always a strong caveat!), this need never matter because the “optimal” (Kelly) stake behind any financial risk is not its Sharpe or Information Ratio, ie expected return over expected volatility. It’s expected return over expected variance, which is time-horizon-independent.

Except profit maximisation under these conditions contains implicit conditions that scare the pants off most normal human beings. You should eg expect at some point to lose more than a third of your wealth with two-thirds probability!

That plus buying every rally and selling every dip levered ~5% tends to put most normal human beings off the mathematically “optimal” strategy (for any set of risk:reward assumptions where the investor actually has confidence).

Improve this answer
$\endgroup$
0
$\begingroup$

Honestly, if you get 50.1% you should be happy:)

Predicting the future is just plain hard and if you can do it reliably then, by definition, you've found a way to make money. Think about how many hedge funds and how many Phd's are working on this right now.

The biggest issues you'll come across are curve fitting and survivor ship bias. ie you'll tune your learning model to the data you have for back testing and what often happens is that after a bit of tuning your model perfectly predicts what happened on the day you test it and that's about the only time it accurately predicts what the market does.

Sorry if that sounds bleak but its incredibly hard to accurately predict the future, Or put another way, success in machine learning for a beginner is probably just not making things worse than random guessing.

Some questions to help flesh out what you are doing...

  • What specifically are you trying to predict?
  • What is your market data source?
  • What ml algorithms are you using?
  • Can they run in real time?

As to what percentage of being correct you should really look to target.... it is a function based on how much you lose/gain for each order you put out.

Improve this answer
$\endgroup$
3
  • $\begingroup$ 50.1% would be nice... I'm just like most of the dopes tat come through here after watching too much TV one day. Yeah, you've seen my kind before. But isn't that how google got started? At least I'm not this guy. $\endgroup$
    – poorly_built_human
    Jun 23, 2014 at 14:30
  • $\begingroup$ ...I can understand the argument of large brokerage firms using AI in conjunction with what they had already been using, but only as a support for their previous assumptions. Maybe I take for granted how many people there are in the world these days. I figured there would x number of capable programmers, and y such programmers interested in pursuing something like this on their own time (where x and y are small numbers). And yet I am still pretty sure there's some 12 year old kid out there who managed to open a brokerage account and is getting 99% returns as I write these very words. Dammit. $\endgroup$
    – poorly_built_human
    Jun 23, 2014 at 14:31
  • 1
    $\begingroup$ 50.1% is not nice for non-HFT frequencies. You have to pay transaction costs, staff to build the system, have drawn this number from a random variable with a large variance (this is just the nature of low frequency data), and do not know the extent of look-ahead bias or other issues that could have biased this figure. Your edge will also deteriorate over time as your competitors discover your alpha. Risk free rate is much better. $\endgroup$
    – user2763361
    Jun 23, 2014 at 16:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.