# What Is A Good Success Rate Using Machine Learning For A Beginner?

I know this question will be quickly destroyed and my account summarily banned, but I just have to ask:

For a trader using machine-learning algorithms (SVMs, ANNs, GAs, Decision Trees) for quantitative finance, without seasoned financial intuition, what would be considered a good confidence / success rate?

I know this will depend on the following, as well as other items I'm not aware of:

-Market / Sector (stocks, commodities, FOREX, etc.)
-Principal investment
-Share price
-News Volatility of sector
-Range of dates used for datasets

Please feel free to list other considerations... But in the end, to make the question crystal clear I'd really like a target number. 75%? At 60% I would be roughly taking 1 step forward for every 10 steps taken. Any less and I might as well flip a coin. If it varies, please list the considerations under which they do so. If possible, it would be preferable to use these models to support trades over a period of days rather than seconds/minutes.

If you have other suggestions for how to go about things based on low/high, principal, markets to consider, share prices, etc. please feel free. If my question does not make sense, please tell me why. Thank you.

UPDATE

-At this point I was simply trying to predict up and down movements over a 5 day period. Simple. 45%.
-Free Yahoo data be my market data source... daily quotes. Wasn't sure if intra-day information would be helpful.
-I've attempted ANNs, SVMs, and some GAs so far.
-I wasn't looking for real-time trading, but instead looking to identify regular tides over a several day period.
-Maybe if I can get my error high enough, I can simply trade opposite my predictions! (no, seriously though)

-
If this post belongs on CrossValidated, then I apologize. Please migrate. Otherwise, please explain a downvote. –  poorly_built_human Jun 23 '14 at 11:51
I think this question is definitely appropriate here. I am no expert on this topic, but I think if you use models like that you aim for a success rate that is well above $50\%$ with the amount depending on the cost of implementing the trading signal so that you place bets with an expected value >0. Further more, you would want to add some additional margin (for estimation and modelling error or changing environment for example). It definitely depends on the cost of getting in and out of the position. Thats why there can't be a single target number. –  vanguard2k Jun 23 '14 at 12:29
I flagged this question for closure. I appreciate your honesty about being a beginner but this question seems to be outside the scope of this SE. Firstly, this question is too broad and opinion-based - as the responses suggest - there's no right answer. I would be OK with a 50.1% success rate at a coin flip and a 0.1% success rate at winning a billion dollars. I'd need to know the transaction costs etc. And even so, there's no final answer because a positive expectancy strategy may be within someone's risk appetite but outside another's. –  madilyn Jun 23 '14 at 18:50

You're thinking about this the wrong way, in my opinion. Win/loss percentage is worthless in isolation. You must consider the symmetry of your winners and losers. You can have a win % of only 40% and still have a wonderful strategy if your your winners are significantly larger than your losers (this is the classic trend follower PnL distribution).

So, you could flip a coin and see 50% prediction accuracy. That would be outstanding if your winners are 2x larger than your losers.

-
Thank you very much, you've been the most helpful so far. I realized that I was not taking into account many factors, but didn't know what they all were. Can you elaborate on your answer a little more before I give you the green check? Is there a specific formula (other than standard risk assessment) for assessing individual trades? I imagine there is a fair amount of game theory involved. Also, is it advisable to map out the movements in multiple smaller increments such that if a move doesn't go as predicted the stop-losses can be activated, or simply hedging bets? ...(continued) –  poorly_built_human Jun 24 '14 at 11:40
...I have read this and this and a few other links on risk assessment, but I'm still having trouble understanding what value machine learning can be if your chances are 50/50. Thank you again. –  poorly_built_human Jun 24 '14 at 11:40

-

Honestly, if you get 50.1% you should be happy:)

Predicting the future is just plain hard and if you can do it reliably then, by definition, you've found a way to make money. Think about how many hedge funds and how many Phd's are working on this right now.

The biggest issues you'll come across are curve fitting and survivor ship bias. ie you'll tune your learning model to the data you have for back testing and what often happens is that after a bit of tuning your model perfectly predicts what happened on the day you test it and that's about the only time it accurately predicts what the market does.

Sorry if that sounds bleak but its incredibly hard to accurately predict the future, Or put another way, success in machine learning for a beginner is probably just not making things worse than random guessing.

Some questions to help flesh out what you are doing...

• What specifically are you trying to predict?
• What is your market data source?
• What ml algorithms are you using?
• Can they run in real time?

As to what percentage of being correct you should really look to target.... it is a function based on how much you lose/gain for each order you put out.

-
50.1% would be nice... I'm just like most of the dopes tat come through here after watching too much TV one day. Yeah, you've seen my kind before. But isn't that how google got started? At least I'm not this guy. –  poorly_built_human Jun 23 '14 at 14:30
...I can understand the argument of large brokerage firms using AI in conjunction with what they had already been using, but only as a support for their previous assumptions. Maybe I take for granted how many people there are in the world these days. I figured there would x number of capable programmers, and y such programmers interested in pursuing something like this on their own time (where x and y are small numbers). And yet I am still pretty sure there's some 12 year old kid out there who managed to open a brokerage account and is getting 99% returns as I write these very words. Dammit. –  poorly_built_human Jun 23 '14 at 14:31
50.1% is not nice for non-HFT frequencies. You have to pay transaction costs, staff to build the system, have drawn this number from a random variable with a large variance (this is just the nature of low frequency data), and do not know the extent of look-ahead bias or other issues that could have biased this figure. Your edge will also deteriorate over time as your competitors discover your alpha. Risk free rate is much better. –  user2763361 Jun 23 '14 at 16:49