How did James Simons clinch that security prices didn't look random?

Question

Source: D. T. Max. Jim Simons, the Numbers King. December 18 & 25, 2017 Issue

[...] In the late seventies, not long after he won the Veblen Prize, Simons founded a small investment firm in an office park near Stony Brook. At the time, he felt stymied by a mathematical problem involving simplexes—a simplex is the polygon with the fewest vertices in any given dimension—and he wanted a break. He tried his hand at currency trading, and then at commodities, but he didn’t enjoy the experience. It was the investing equivalent of wet-lab work. “It was fundamental trading, not systematic,” he said. “It was very gut-wrenching.” He felt that there must be a more statistical way to make money in the market. “I looked at the price charts and analyzed them, and they didn’t look random to me,” he says. “They looked kind of random, but not completely random. I felt there had to be some anomalies in this data which could be exploited.”

Not specializing in probability or statistics, did Simons spot something that the outstanding probability or statistics specialist professors didn't?

If so, how? What spurred him to the conclusions that I bolded overhead?

Answer 1

I'm sure Simons, as a first-rate pure and applied mathematician, had sufficient understanding of statistics to detect market inefficiencies and anomalies. As far as I know, the development and practice of statistical arbitrage as well as derivatives pricing has never been the exclusive domain of "outstanding probability or statistics professors."

Along with his first partners, Simons developed strategies to exploit such anomalies. I believe he has said himself in interviews that the early strategies were primarily trend-following. As exploitable anomalies do not persist indefinitely, he clearly moved on to other things as his firm and computing power evolved -- looking at higher-frequency effects, larger data sets, and more advanced forms of pattern detection and signal processing.

Also he was, by no means, the first to notice and exploit non-random price behavior. He just became extremely good at it and built a large hedge fund with a deep pool of talent to keep the ball rolling.

For example, Edward Thorp, was probably one of, if not the first, to notice mispricing in convertible bonds in the late 60s and early 70s -- developing convertible bond arbitrage. He moved on to other forms of statistical arbitrage as well. His book Beat the Market (1967) clearly shows he discovered the essence of the Black-Scholes-Merton model well before it was published in 1973.

Victor Niederhoffer, another well-known hedge fund manager for a variety of reasons, studied many anomalies in the daily return patterns of S&P 500 stocks, again around 1970. He worked with Frank Cross who examined returns over the period 1953 to 1970. As described in The Education of a Speculator, it was observed that the index rose on 62.0 percent of Fridays but only on 39.5 percent of Mondays. The mean Friday and Monday returns were 0.12 percent and -0.18 percent, respectively. The probability that such a difference would occur by chance is less than one in a million. Many such event studies can be found in the literature and generally appear, if published by practitioners, well after the anomaly has been exploited.

Answer 2

Jim Simons' initial intuitions about nonrandomness were probably driven by the very psychological/evolutionary predispositions to want to find the hidden meaning within noise that affect humanity in general. That Jim Simmons has been effective is a more a testament of his abilities and timing rather than his inclination to clinch that some patterns were not random.

From what the public knows about Simons and his key hires, RenTec’s core competency is in discerning pseudorandom from the truly random. Pseudorandom processes appear to be random but actually are partly comprised of non-random patterns. That the market can be modelled as a series of pseudorandom processes is actually not inconsistent with most forms of the Efficient Market Hypothesis.

Simons’ earlier experiences as a cryptological code breaker were formative of his ability to see the early applications of quantitative signal processing techniques to markets. Early hires at RenTech were experts in signal processing and pattern recognition. Moreover RenTec was probably the earliest market participant to adapt highly sophisticated quantitative analyses (e.g., deconvolution, Bayesian filtering and calibration, speech recognition, natural language processing, etc) to markets. For example, Leonard Baum was an early hire at RenTec whose eponymous Baum-Welch model is intended to detect and calibrate hidden Markov Models.

So the fact that Simons clinched that prices were non-random is not unique. Rather, it was his ability to see the early applications of technology that made his initial intuition unique.

The New Yorker article elaborates:

Jim’s genius was in seeing the possibilities for quantitative trading long before others did and in setting up a company in which he provided outstanding scientists with the resources, environment, and incentives to produce.

But perhaps we shouldn’t expect the same performance going forward. RenTec’s early success was during a time when market efficiency was relatively much greater. Because RenTech's edge depended on pushing into the frontiers of inefficiency, its alpha decay is very real. In 2012, I was told that the cumulative total of all RenTech's signals have lost on average about $\frac{3}{4}$ their original predictive power since inception (or was it that $\frac{3}{4}$ of all signals had lost all of their predictive power???). (I wonder what percentage never had any predictive power in the first place... alas, all models are wrong, even if some are useful).

Moreover, while his recognition of the potential for quant analysis made him unique early on, it seems that his ability to recognize its limitations makes him unique going forward. Judging from recent interviews, Simons fully recognizes the real world competitive challenges of increasingly sophisticated arbitrage mechanisms and, thus, the need to stay ahead of the technological and knowledge power curves. Simons amplifies:

Trend-following is not such a good model. It’s simply eroded... Statistic predictor signals erode over the next several years; it can be five years or 10 years. You have to keep coming up with new things because the market is against us. If you don’t keep getting better, you’re going to do worse.

Answer 3

I will disagree with RPL's answer - Simons is not particularly known as an applied mathematician, but he did work for some time at the Institute for Defense Analysis [IDA] (he was fired for insubordination), which is a signals intelligence shop. The game in signals intelligence is to find the signal in the noise, and that is exactly the game in quant finance - it is not an accident that Simons' successor at the head of Medallion is Bob Mercer, who comes from the IBM dictation project.