How to evaluate a success rate of a trading strategy

Question

In order to compare various trading strategies, I am trying to calculate the success rate (the ratio of winning and losing trades).

While it is clear to me that this indicator is far from being an accurate reflection of the strategy's strengths, leaving out the time horizon and relative size of the trades from the picture, I am particularly concerned by the following problem:

For a strategy that makes simple trades buying a stock at once, then selling it later at once, it is straightforward to identify the winning and losing trades and calculate the ratio.

However, if a strategy chooses to accumulate a stock gradually over multiple transactions, and/or sells them gradually, the notion of a trade becomes somewhat flaky. In particular, it becomes hard to compare strategies like A and B where

A buys 10x @ \$100 and sells 10x @ \$150 (1 winning trade with 50% profit)

B buys 5x @ \$90 and later 5x @ \$110, then sells 5x @ \$140 and later 5x @ \$160

Intuitively, if time gap between the individual buy and sell trades in the example B is small, the strategy made the same decision (to buy 10 pieces of the stock) and ended up with the same outcome (total profit of 50%).

The more granular the single transactions are, the more difficult it seems to support this intuition with some kind of a rule that would group the individual transactions into trades based on their proximity, so that a single trade corresponds to a single decision made by the strategy.

Is there a standard way to solve this problem or can you point me to any references that are at least in some way related ?

EDIT: A couple of clarifications to address the comments below:

• I'm treating a strategy as a black-box here and only want to evaluate its historical performance. I have no way of back testing the strategy in a different environment, which is also not my intention - the only thing I'm looking for is calculating the success rate to get an (admittedly, somewhat biased) picture

• The example of a worst-case scenario is a strategy that is buying and selling a single stock X, thereby just changing weights in the portfolio composed of X and cash. It would generate lots of transactions (selling and buying according to the price fluctuations of the stock X), but I'm not sure how to group these into "trades" or "decisions", so that I can calculate a ratio between wins and losses. If I simply define a trade as happening between the first purchase and the last sell of a particular stock, this strategy would only have made a single trade. I know this is a somewhat contrived example but I have the same problem in less extreme cases when the strategy is rebalancing or repeatedly trading non-unique stocks and I'm trying to compare it to strategies that trade a large number of unique stocks.

Answer 1

I don't know if there is a standard way of solving the problem, but I solve it thus:

1. Strategy A bought for $C_a$ dollars and sold for $S_a$ dollars for a result of $R_a = S_a - C_a$ over $T_a$ days.
2. Strategy B bought for $C_b$ dollars and sold for $S_b$ dollars for a result of $R_b = S_b - C_b$ over $T_b$ days.

Where

1. $C_a$ and $C_b$ is the total sum of investments, independently of whether you bought once or several times. Same goes for $S_a$ and $S_b$ with respect to sales. Reinvesting increases both terms and thus is neutral with respect to $R_a$ and $R_b$.
2. $T_a$ and $T_b$ is the duration from first purchase to last sell - if you keep an investment beyond the end of that duration calculate as if you would sell it at the end of the duration. (Sales of investments that were already held before the start of period may not be considered in the calculation.)

To make the strategies comparable we need to normalize investment and period. Hence we set

1. $C_b' = C_a$
2. $T_b' = T_a$
3. $R_b' = R_b\cdot \frac{C_a}{C_b} \cdot \frac{T_a}{T_b}$

Now you can compare $\frac{R_a}{T_a}$ and $\frac{R_b'}{T_b'}$: whichever is greater is the better strategy.

So given your example (assuming $T_a = T_b$):

1. $C_a = 10\cdot100\$ = 1,000\$$ and $S_a = 10\cdot150\$ = 1,500\$$, hence $R_a = 500\$$.
2. $C_b = 5\cdot( 90 + 110)\$ = 1,000\$$ and $S_b = 5\cdot(150 + 160)\$ = 1,500\$$, hence $R_b = 500\$$.

Hence in your case both strategies would be equal (assuming $T_a = T_b$).

Oh, and it's also important to vary only one parameter out of (in this case)

1. strategy
2. starting date
3. market place

For example, if you want to compare two strategies A and B then you better don't test A in the US and B in Europe or run A during January 1st 2008 - January 1st 2012 and B during January 1st 2009 - January 1st 2013.

Similarly you need to check the same strategy in the same marketplace at different times or at different marketplaces at the same time to find out whether the success of this strategy was just luck or whether there might be something to it.