Say I have found a way through technical analysis to predict how stocks would behave with 58% accuracy, how good is this percentage?
In the long term you will underperform buy & hold because you need an accuracy of at least 65%.
See these papers for more:
- Bauer, R.; Dahlquist, J.: „Market Timing and Roulette Wheels Revisited“, CFA Institute, 2012.
- Sharpe, W.: “Likely Gains from Market Timing”, Financial Analysts Journal, March/April 1975.
It's not bad but you have to backtest the method out-of-sample.
Say you have discovered an indicator that works 100% in history, you still cannot be sure if it works next time.
Another advise is you might want to investigate the distribution of loss when your system fails to work. If your system delivers 1% every time you trade, and loses 10% each time it fail, you probably wind up losing in the end.
Directional forecast is insufficient. You could have a signal that has 100% accuracy and you would not necessarily be able to profit from it because of transaction cost, implementation etc.
It's not unusual to find a financial time series with positive trend samples biased between 55-60%, depending on the period sampled. Stocks tend to have an upward drift over the long run. When you account for the drift, I would say, that number is really not much better than chance.
A better way to verify your question would be to make certain to build your estimates and models (with parameter optimization) on a certain portion of the sample and comparing out of sample results of your model's performance to the actual performance of the stock series under the same out of sample time frames. Using a rolling window is commonly used.
And also, make sure to look at the actual expectation or performance of the actual returns over the comparison period, as simple directional signs can be close to meaningless without values for context.
Incidentally,this researcher tested 1 million machine learning models using data from 93 - 2008 to predict SPY ETF direction and acheived an astounding 78% in sample performance. When the trained model was tested over the next two years out of sample data, it dropped all the way to 50%... not so remarkable, and a good example of why its really important to consider validating data over different periods.