28
$\begingroup$

One of the answers to my previous question regarding the strategy of Renaissance Technologies, there was a reference to The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It.

After doing some browsing in the book I found that it states that Renaissance Technologies obviously very successfully employs cryptography and speech recognition technology for forecasting financial time series.

Do you know of any good papers (or other references) where the use of either of these technologies in connection with finance is shown?

$\endgroup$
2
  • 3
    $\begingroup$ Maybe these two posts can interest you: intelligenttradingtech.blogspot.com/… $\endgroup$
    – Zarbouzou
    Commented Apr 18, 2011 at 14:47
  • $\begingroup$ @Zarbouzou: Seems to be an interesting blog - I added it to my list. Thank you! $\endgroup$
    – vonjd
    Commented Apr 18, 2011 at 15:02

8 Answers 8

29
$\begingroup$

By "cryptography" you mean information theory. Information theory is useful for portfolio optimization and for optimally allocating capital between trading strategies (a problem which is not well addressed by other theoretical frameworks.)

See:

--- J. L. Kelly, Jr., "A New Interpretation of Information Rate," Bell System Technical Journal, Vol. 35, July 1956, pp. 917-26

--- E. T. Jaynes, Probability Theory: The Logic of Science http://amzn.to/dtcySD

--- http://en.wikipedia.org/wiki/Gambling_and_information_theory

In the simple case, you would use "The Kelly Rule". More complicated information theory based strategies for allocating capital between trading strategies take into account correlations between the performance of trading strategies and the relationship between market conditions and strategy performance.

As for Natural Language Processing and speech recognition; when you examine the founders of Renaissance Technology, you will notice that many of the early employees had backgrounds in natural language processing. Naively, you might assume that RT is using NLP based strategies.

However, you will find that all of RT's NLP related hires have backgrounds (published research, Phd thesis's) in speech recognition and specifically in Hidden Markov Models and Kalman filters. The academic background and published research of RT employees gives you a good idea of the algorithms they are using.

The information that has leaked out of RT suggests that RT heavily uses "hierarchical hidden markov models" for latent variable extraction from market time series. It is also believed that RT has developed a proprietary algorithm for "layering" multiple trading strategies for trade signal generation.

RT does not have a single secret trading strategy that magically generates billions of dollars a year. Renaissance Technology's trading strategies are based upon the integration of information from multiple mathematical models.

$\endgroup$
3
  • $\begingroup$ A good paper on the Kelly criterion is Ed Thorp's The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market. He derives the Kelly formula for two simultaneous bets on page 19, and on the following page talks about applying it to assets with correlated returns. I haven't been able to work out a closed-form solution, but it's fairly easy to generalize the method to an arbitrary number of assets by using a Monte Carlo approach. $\endgroup$
    – joshayers
    Commented Apr 27, 2011 at 20:41
  • $\begingroup$ One more comment - Ernest Chan's Quantitative Trading has a chapter on the Kelly criterion in finance. Unfortunately, I found it to be a very shallow treatment. He just presents a formula with no real explanation or derivation. $\endgroup$
    – joshayers
    Commented Apr 27, 2011 at 20:43
  • $\begingroup$ By cryptography I don't mean information theory but crypto-analysis, so extracting hidden patterns out of streams of symbols - but anyway: Good answer, thank you! $\endgroup$
    – vonjd
    Commented Apr 29, 2011 at 13:20
13
$\begingroup$

In a very, very general sense, what Renaissance Technologies does well [and others try to do, many do less well] is understand where the "true" signal is (i.e. where prices should be) and what is noise (i.e. over-/under-reactions by others in the market) in the total signal of market prices. Generally, trading profits are made by taking the opposing position to someone who over- / under-reacts to where the market price is going to be because the market will "come back" to the true price.

Cryptographic algorithms and speech recognition algorithms have been developed to accomplish essentially the same thing ... it's necessary to separate noise from underlying signal in those and other applications of information theory, basically this is machine learning ... in general, becoming proficient in machine learning is skill that applies in many fields and for the mathematically-inclined individual, it is well worth studying [because there are so many applications, not just trading]. A good starting point would be the lectures by Andrew Ng of Stanford University

Attempting to beat the hedge funds really isn't something that should be copied by amateurs, i.e. it's an expensive way to get an education ... the algorithms are dynamic; the people using these algorithms must be faster, better, smarter than others who are using these algorithms ... continually updated, adjusted, advanced, accelerated, sharpened by teams of very smart people ... Renaissance isn't alone; it has smart competitors; there's a constant struggle by well-capitalized new firms with smart founders to make these adjustments with smarter, brighter, more capable teams of people who have access to better, faster technologies ... for example, many these algorithms must now execute so rapidly that it's necessary to use specifically-designed hardware that use ASIC integrated circuits or similar hardware, i.e. software on a supercomputer isn't fast enough to execute the trades ... the oscillation of the noise around the signal happens faster and faster.

Generally, the world benefits by the development of these algorithms and hardware, because the advances in this technology can eventually "spill over" into other projects in the rest of the world ... you aren't likely to find out what Renaissance Technologies and its competitors are doing primarily mostly because the people doing it don't have time [while they are doing it] to write papers and explain; it's not really that they want to keep secrets (i.e. there's some advantage to having your competition look at what you were doing last month, last year because by the time they understand it, you already know how to beat your old strategy), it's mostly that they don't have time / inclination to write up the explanation just yet ... as with all pursuits, the day eventually arrives where a person just doesn't really care any more and maybe wants to answer questions or tell the story.

Anyone who smugly tells you that they can't tell you because it's a secret basically is telling you they're just somebody like Mr. HR-clerical-job-app-screening-person and they couldn't begin to understand -- so it is easier for them to tell you that it's a secret. It's not that much of a secret ... mostly, it just happens fast ... so you need to be able to pick it up on your own without somebody explaining ... if you need to learn by having it explained, you shouldn't try to use it.

$\endgroup$
12
$\begingroup$

I honestly think that most people who could be able to answer to this question simply won't either because they actually work for Renaissance, or because they work in a top quant hedge fund and they'll keep it a secret.

I discussed this topic once during an interview and the guy said "we'll discuss this further if you get the job" lol.

About papers, I'm sure that there have been incredible findings in the industry which are just kept secret until they are found by academy, or simply become obsolete....

And honestly, it's quite fair...

$\endgroup$
2
  • 2
    $\begingroup$ Fair enough. What confuses me is that obviously nobody even tried to do research on these topics. If it is so well known by obviously many that this is an important ingredient of the secret sauce then why doing the one hundredth paper on moving averages or whatever...?!? $\endgroup$
    – vonjd
    Commented Apr 19, 2011 at 12:58
  • 4
    $\begingroup$ @vonjd: I guess it's because these subjects are extremely complicated and difficult to tackle. It is my understanding that they required some of the greatest minds of their generation just to get started. Now, if you start digging into it and eventually get to understand the matter, you'd be a fool to actually explain other people how it works. Make the profit, take the money, and get yourself a private island. $\endgroup$
    – SRKX
    Commented Apr 19, 2011 at 13:05
9
$\begingroup$

I have been learning more about speech recognition motivated by its application to financial forecasting. I have identified a couple connect points. Turns out each of these tools can and are regularly used in financial modeling as well.

  1. Use of Markov Models
  2. Use of Fourier transforms (sine/cosine decompositions)
  3. Use of component analysis
$\endgroup$
5
$\begingroup$

The Brown et al. paper and its connection with trading is discussed here: http://jochenleidner.posterous.com:80/from-speech-recognition-to-trading (mirror)

$\endgroup$
2
  • 4
    $\begingroup$ The link is no longer active, can you please post an alternative link if it is available $\endgroup$
    – coder_bro
    Commented Jul 9, 2013 at 10:37
  • $\begingroup$ @Ngm fixed~~~~~ $\endgroup$ Commented Jul 16, 2017 at 22:42
5
$\begingroup$

Speech recognition signal processing is complex and possibly similar to the complexity of financial markets. They are similar as per characterictics the non stationarity, noise types and other aspects such us the existence of a spectrum etc conceptual frequency and the grammar to construct and articulate concepts is not evenly and randomly distributed; so pattern discovery can occur and therefore signal processing methods may be similar.

In language processing evolution of speech incorporates new words and old words are eliminated. In this respect financial markets are the same where signals of certain characteristics appear and have limited life spans. Once a person at Renaissance mentioned to me that since the early start of the models today only 25% of the original signals are being used while 75% of the pass signals used have lost all predicting power. This is the same as saying that 75% of the patterns encountered in the past which had statistical significance in relation to returns have since vanished. So any model if fixed as per the patterns to encounter is deemed to fail after a few years of using it. Maybe Language processing as per financial prediction is related to the discovery of new patterns which are formed or derived from previous existing patterns.

Cryptology may be in the techniques employed as to pint point the true signal from noise types and in this respect a decryption algorithm will be the equivalent of having the mechanism to automatically transform an encrypted signal into a decrypted one. Therefore is related to denoising.

$\endgroup$
3
$\begingroup$

As we all know, it will not be in the public unless academical research. However there is a clue. We can search for the paper that current CEO of Renaissance published. That can be worked as a stepping stone. In my opinion, voice recognition algorithm was used to eliminate noises to see the clear pattern. For specific voice to be recognized, it has to eliminate a lot of noises. Let's share the papers if anyone find written by current CEO of Renaissance. Simons has retired and the current one is the one with voice recognition, if I am not mistaken.

$\endgroup$
1
  • 3
    $\begingroup$ Could you provide us with a link to the article please? $\endgroup$
    – SRKX
    Commented May 7, 2012 at 12:20
2
$\begingroup$

Here's a $40 million Twitter-based hedge fund from Derwent Capital Markets:

Twitter-hedge fund article

My guess is, you'll get a wide swing of opinions on this scheme.

Edit 1 (05/25/2012) =================================

Here's an update on the "Twitter Fund":

Update on Twitter hedge fund

No big surprise.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.