I have developed a trading algorithm, surprisingly simple in nature (I did start off with grand plans of applying Machine Learning to this problem as I am a data scientist by trade).

I would place the code here, but I would like to do some appropriate backtesting before throwing it open for peer review. In any case, I am seeing accuracies of predicted_trading_signal compared with perfect_insight of $+80\%$ across many of the russell 2000 group, which as a data scientist, and a rational person seems too good to be true. To this end, I would like to know if this is an appropriate test

Here is some pseudocode:

# generate trading signal up onto data[i]
predicted_trading_signal = some_algo(data[:i])

# generate perfect insight signal
perfect_insight = another_algo(data[i], data[i+1])
# where data[i+1] is say the next day
if some_statistical_property_of_data > lambda:
    if  predicted_trading_signal == perfect_insight:

# after running over all windows of interest for a particular stock 
accuracy = 100.0* correct /float(correct + incorrect)

I wish I could be more concrete with the code!

  • Is this a valid approach to computing accuracy
  • is conditioning on information up until todays date and using the following days price typical in testing strategies without data leakage (a subtle problem to a naive person such as myself in trading)
  • Is there a better approach?

My final question (hopefully this is considered part of the bigger question - I hate multiple questions in one post on mathstackexchange)

Can I determine the profit-loss calculation as $$ P_i = \text{trading_signal}_{i-1}\left(S_{i} - S_{i-1}\right)\cdot C_{i-i} $$


  • $C_{i-1}$ is the initial capital invested.
  • $S_i$ stock price
  • $\text{trading_signal}_{i-1}$ is initial trading signal

To begin with, I'm assuming that your trading strategy gives output something of the kind - Buy, Sell, Neutral

While on a cursory glance, your calculation seems fine, here are a few things that you can look at

  • Why does the perfect insight only look at the incremental change? Many trading ideas are path dependent - so re-check if your best suited action would have changed if you added history.
  • The accuracy parameter, as I understand is the percentage of wins in the strategy trades. The more commonly used term for this is "hit ratio". However, the hit ratio is a misleading number. The hit ratio should always be look at, in conjunction with the "Average Winning Trade / Average Losing Trade" number. Only then can you arrive at the expected values. What can happen is that you have a strategy which has an 80% hit ratio, but your average win/average loss is 0.1, which would mean that overall, the strategy is generating negative returns.
  • I see that you are using a statistical condition on the data - I hope that this accounts for the transaction costs - as in, the trade is called a win only if it is sufficiently higher than the entry price.

As for your query on Profit calculation, you have missed out on the dividing factor.


  • $\begingroup$ Thank you for getting back to me. Ah, thank you for putting some actual terms to the metrics i am using. What do you mean by path dependent? As my only knowledge of this comes from option pricing methodologies? Finally, the statistical property is a condition based on how far from the previous value, so sort of encodes the buy sell transaction costs (sort of). $\endgroup$ – Chinny84 Jan 19 '17 at 13:27
  • $\begingroup$ It was great insight to include the avg_winning_trade/avg_losing_trade and it definitely dropped the performance on a 40% of the stocks that I am looking at. If you could edit with some reference to the path dependent analysis then that would be appreciated! Cheers $\endgroup$ – Chinny84 Jan 20 '17 at 4:30
  • $\begingroup$ It is your call if the path affects your strategy. For example, there are pattern trading strategies (very simplistic example), where the trade (and the perfect insight), depends not just on the current day return, but the history also (the pattern is formed over a number of days, so looking at just the incremental data alone, i.e, (data[i], data[i+1]) will not give you much insight, instead, you would need to look at data[:i+1] Have a look if your strategy is affected by the path, and if it is, tweak the perfect insight parameter accordingly. $\endgroup$ – Rehan Jan 20 '17 at 6:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.