Proof that the number of trades done (successfully) matters for whether or not a strategy was lucky

Question

Me and a friend is trying to settle and argument in relation to the following quote by Nassim Nicholas Taleb:

I don’t want to spend too much time on Buffett. George Soros has 2 million times more statistical evidence that his results are not chance than Buffett does. Soros is vastly more robust. I am not saying Buffett doesn’t have skill—I’m just saying we don’t have enough evidence to say Buffett isn’t doing it by chance."

I told him I would rather go with a guy who did 10 000 small good calls than 10 large good calls given that their return history was the same, to which he replied that this reasoning has no basis in maths, finance logic etc. Is he right, or is there a rigorous way of showing that a strategy that has a larger sample size of "decisions made" is more robust to potentially being lucky?

Sorry if this is off-topic.

Answer 1

Of course there is a rigorous way to prove that with larger sample sizes the sample statistic converges to the population statistic, if you pick the mean then the average value of a sample set will converge to its expected value over larger and larger sample sizes. Take a dice. Take a die, denote 6 as best and 1 as worst. Then roll it 10 times, then calculate the average face number you rolled across the 10 rolls. Let's say you got a 5.xx. So, does this mean the die is not a fair die because you scored so high? Maybe, maybe not. You are not sure, luck may be in place here. It actually most certainly is. Now roll the same die 1000 times. If you don't get an average close to 3.5 you can be almost certain that the die is not fair. You can now adjust your confidence interval the larger you make the sample size. Pretty easy and basic concept imho. Hope it makes sense. Here the proof:

Take a look at the concept of the law of large numbers: http://en.wikipedia.org/wiki/Law_of_large_numbers

The proof is on the same page and directly relates to your problem.

By the way, I concur with Nassim Taleb on this one (though I disagree with his style of trading and some of his opinions on options trading). George Soros has proven many more times than Buffet that his investment and trading skills are highly skilled and that his returns were not the outcome by mere luck. Its that most people only know about his BOE play but hardly anything else. Buffet in turn has done a lot bigger deals but fewer in between, simply because his investment style is much more longer term oriented than Soros's.

Answer 2

You can point out to your friend that, statistically speaking, having more observations reduces uncertainty in estimators.

Mathematically, $SE_\bar{x}\ = \frac{s}{\sqrt{n}}$, showing that the standard error of a statistical estimator decreases with increased observations.

This argument is concise and consistent with the Taleb quote.

From wikipedia on standard error.

• a simple way to think about it is to imagine you had to determine if a coin was un-biased or not. Would it be more robust to draw conclusions over 10,000 (empirical) flips or only 10?

Answer 3

His friend is an very serious and high level former physicist turned quant and he'll tell you for free that this is not a debate about the law of large numbers (which was last interesting before I hit puberty) or any of the other elementary statistical explanations listed above - none of these things are relevant or revealing. It's essentially a debate about the nature of a "trade".

The original debate was whether Jim Simmons or Warren Buffet was a better investor, and the crux of the debate came down to my good friend asserting that Simmons was a better (read: less lucky / more "skillful") investor because he had done a (much) greater absolute number of trades than Buffet over the course of his career, despite Buffet's career being substantially longer (almost double the length). This is because Renaissance tend to run computer driven trading strategies that are often very high frequency and use complex and quick dynamic portfolio rebalancing techniques based on live, noisy, incoming signals - lots of very small and funny trades with a very short life span and no objective purpose other than because the computer deems them necessary as part of its greater objectives. The majority of these trades, on their own, do not even intend to be profitable - rather, they're geared towards preventing potential unprofitability.

Buffet, on the other hand and as we all well know, does infrequent and enormous trades based on months of fundamental research. These positions are usually held in perpetuity (he has been a major stockholder in Coca Cola for more than 20 years) and are often intended to maximize dividend and other cash flow income, although he is also fond of distressed opportunities as well (LTCM). Consequently, it should be exceedingly obvious that Buffet does far fewer trades, but all of them are intended to be profitable.

The assertion that this makes him less skillful than Simmons (or Soros, which is an entirely different kettle of fish completely and one only brought up in said debate because my good friend revers Taleb to an unhealthy degree and believes anything he has said can be used to crush an argument) is completely ridiculous. I really shouldn't have to explain why, but my essential point is that 1 trade does not directly equal another. Me buying shares in Anglo American on my dinky little online broker is in no way reminiscent of RBS bundling up 10,000 mortgages and selling the multi-billion securitization to an SPV - yet both are single trades. So applying the law of large numbers to the absolute number of trades done by these titans of the game over their careers (and thus assuming all trades are equal) to establish luck is really, astonishingly, stupid. Its the same as saying all pieces of string are the same length. A more intelligent appraisal would be to look at absolute returns, drawdowns, running costs and receivables. Both have delivered outrageous returns for a very long time, though Buffet has done so for far longer with fewer, smaller drawdowns and at considerably less expense. Additionally, if Simmons turns off the machine tomorrow, profit will instantly become negative and stay so in perpetuity - he has no receivables. Buffet hasn't had to get out of bed for 30 years and he would still earn hundreds of millions for himself and his shareholders in dividend income alone, not counting his other cash streams. Performance wise, they are historically comparable, but it is worth pointing out that performance wise Buffet beats Soros into the dust in every metric you care to pick, except the breadth of securities he traded and the number of absolute trades he did. Some of his misses and drawdowns are gargantuan, and it does surprise me that he never blew Quantum up at one stage or another.

The notion that he was in any way lucky over the course of his 50 year career as the most successful investor who has ever lived is so preposterous that it speaks to an astonishing lack of common sense, and the application of the law of large numbers to this comparison has no basis in mathematics, logic or finance. This debate is ultimately about fundamental vs quantitative analysis, and if there was a clear winner in that debate one or other side would cease to exist quite quickly.

Answer 4

I think Taleb oversimplifies the concept (very unusual for him, since he likes to make things unnecessarily complex). It's a much more complex issue then it appears on the surface, as it touches on number of questions - actual returns, time at risk, percent winners, etc.