28
$\begingroup$

I'm writing my master's thesis about stock price prediction using machine learning methods. During my literature review, I noticed that a lot of research produced on this topic is of poor quality, published in non-finance related journals or unpublished/peer reviewed alltogether. There is no paper to be found in leading journals like journal of finance or journal of financial economics on the topic.

I'm curious as to why this is the case. Did the academic world move on, and simply accept that markets are generally efficient a long time ago? Or are the leading journals overlooking a key technique that could effectively forecast stock price?

Improve this question
$\endgroup$
7
  • 7
    $\begingroup$ Out of curiosity: is there any evidence that papers published in top journals produce better risk-adjusted returns than those published in lesser journals?! : ) $\endgroup$
    – user42108
    Mar 16, 2021 at 12:49
  • 22
    $\begingroup$ If somebody develops a price prediction method that actually WORKS, they are not going to waste their time writing papers for academic journals. $\endgroup$
    – alephzero
    Mar 17, 2021 at 17:39
  • 7
    $\begingroup$ The people who do this are doing it to make money. They can only make money while their algorithm can exploit a gap in the market. Publishing the strategy to identify and exploit said gap will cause other people to also do this, closing the gap, and permanently removing said opportunity from the market, and ultimately rendering the technique only marginally profitable. There is no motivation for the person who discovers such a gap in the market to share their profit with others until the gap is fully exploited and closed to a point of marginal profitability. $\endgroup$
    – J...
    Mar 18, 2021 at 10:44
  • 2
    $\begingroup$ Because machine learning does not work very well in fat-tailed domains due to insufficient samples of the key events to learn from. $\endgroup$
    – crobar
    Mar 18, 2021 at 13:59
  • 4
    $\begingroup$ I hope others here will find this humorous (please do not get insulted by this if you're in academia, I love research myself and try to publish papers). The old joke goes: if you're good at something, you do it. If you're not so good at it, you at least write books about it. And if you're not good enough to write books, at least you teach it :). If someone is making money using machine learning, they'd fall into the first category, not the second or the third :) $\endgroup$
    – Jan Stuller
    Mar 19, 2021 at 11:37

10 Answers 10

Reset to default
38
$\begingroup$

I think you're overlooking a third explanation:

Nobody that found a successful technique to generate alpha has published it. I can think of the following causes:

  1. If you're an academic, why share your brilliant idea?
  2. These techniques require a lot of data and financial data can be expensive, researches that work at firms that have access to this data don't share their findings with the public.
  3. Academics did find a lot of signals already the old fashioned way.

Despite this, fancy techniques such as AAD and Reinforcement Learning are discussed publicly. These methods don't generate any alpha however.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Reinforcement leanings generates “no alpha” sounds a bit reductive tbh. Even if the trading strategies directly have not found a consistent approach to alpha generation (which I doubt), RL supplements trading in numerous ways, from best execution to more robust optimisation. If someone is saving millions in slippage I’d say RL was pretty instrumental with the alpha there. $\endgroup$
    – Dhruv Mahajan
    Mar 16, 2021 at 11:21
  • $\begingroup$ @DhruvMahajan Thanks for the paper I linked in this answer. I think you make good points in general but since the question mentions "effectively forecast stock price" I only focused on that. $\endgroup$
    – Bob Jansen ♦
    Mar 16, 2021 at 12:38
  • 16
    $\begingroup$ Another reason not to publish: because of the feedback loop where investors react to the market but the market also reacts to investors, if an algorithm works and everyone starts using it, it stops working. $\endgroup$
    – BlueRaja - Danny Pflughoeft
    Mar 16, 2021 at 20:06
  • $\begingroup$ @BlueRaja-DannyPflughoeft: And even from an academic perspective, a model which describes how things would work if the model weren't published would be of only marginal interest, since it would be impossible to validate or usefully expand upon. $\endgroup$
    – supercat
    Mar 16, 2021 at 22:02
  • 3
    $\begingroup$ "If you're an academic, why share your brilliant idea?" Because that's what academics are expected to do. "publish or perish" $\endgroup$
    – Barmar
    Mar 18, 2021 at 15:11
30
$\begingroup$

In the early 2000s I met the Quant Team at Barclays Global Investors in San Francisco and I tried to convince them to submit some of their research to the journal I was managing at the time, Quantitative Finance. This was met with some mirth and incredulity and they told me that they might consider submitting some of the work that went into v2 of their model but that actually they were working on v9. It hadn't really dawned on me until this point that the academic literature only scratches the surface of the proprietary research that is being conducted in institutions world-wide.

Improve this answer
$\endgroup$
4
  • $\begingroup$ Thank you for this interesting perspective. I was thinking about programming such an ML myself. But I think I'm better off with a passive broad-based indexing strategy. $\endgroup$
    – Psychotechnopath
    Mar 16, 2021 at 20:52
  • 13
    $\begingroup$ By now they will admit that v9 was a bunch of nonsense, but no way would they would publish the magic in v23. $\endgroup$
    – steveo'america
    Mar 16, 2021 at 22:55
  • $\begingroup$ And do you believe that v1-23 were able to generate alpha, or was the effort fruitless to begin with because all information that can be used in such models is already priced in the stock because markets function efficiently? $\endgroup$
    – Psychotechnopath
    Mar 17, 2021 at 9:48
  • $\begingroup$ @Psychotechnopath Stock markets do not function efficiently, but can appear to do so for periods because of the fat-tailed nature of their statistical distribution. The sparse nature of important events (e.g. crashes) makes learning difficult (for humans too!). $\endgroup$
    – crobar
    Mar 18, 2021 at 14:01
25
$\begingroup$

There are very many finance papers using machine learning

This includes asset pricing and stock returns

But ultimately, it's just not their job

  • Top finance and economics journals do not exist to discuss the best trading strategies that exist for a few months and then vanish again. That's just not the point of financial economics. Look at cross-sectional predictability: it's very easy to come up with factors that predict returns. But the entire debate in top finance journals is about what economic fundamentals explain why and how stock returns are predicable. Essentially, you have an easier time getting into an academic journal with estimating a general equilibrium model that explains why stock returns behave the way they do rather than with fitting some model that gives you an alpha for some time (without any economic meaning).
Improve this answer
$\endgroup$
5
  • $\begingroup$ Thank you for this answer! You say "Many academics agree that US stock markets are pretty efficient and predicting individual stock returns (in a time series sense) is very difficult, even with machine learning. However, there are plenty of other applications where machine learning (as yet another statistical tool) is very useful". This is probably the reason why I couldn't find "good" material; was looking specifically for such classical prediction papers. Am sure ML can be of help in other ways! $\endgroup$
    – Psychotechnopath
    Mar 16, 2021 at 12:49
  • 1
    $\begingroup$ @Psychotechnopath The problem with predicting single stock returns (say on a daily basis) is the enormous amount of noise that you need to filter out - be it with machine learning or classical statistics. It's a very difficult task. Nonetheless, people of course work on this and I hope I could cite a few papers that look interesting to you :) $\endgroup$
    – Kevin
    Mar 16, 2021 at 12:52
  • 1
    $\begingroup$ Just out of curiosity; if EMH is valid; wouldn't the search for such models make markets more efficient? E.g. when my hypothetical perfect ML model predicts with 100% certainty a price increase of a certain share with say; 30 dollars I would buy shares as long as the price has not reached the 30$ increase... The act of buying would increase share price. $\endgroup$
    – Psychotechnopath
    Mar 16, 2021 at 12:55
  • 1
    $\begingroup$ @Psychotechnopath That sounds a bit like technical analysis. But in principle yes: people reacting to news and trading upon them contributes to ``price discovery'' and is important to find an equilibrium price. The question of EMH is whether you algorithm that has been trained on past data can predict future returns. In an efficient (and perfect) market, all past information has already been priced and only new (unexpected) information can move prices. The question is whether your ML algorithm correctly predicts how a stock price reacts to new information. $\endgroup$
    – Kevin
    Mar 16, 2021 at 13:04
  • 2
    $\begingroup$ Strongly agree with this statement "finance and economics journals do not exist to discuss the best trading strategies...". $\endgroup$
    – nbbo2
    Apr 1, 2021 at 16:54
8
$\begingroup$

Adding to the answer of @BobJansen there are some additional worries with complex machine learning models (eg. Neural Networks of any kind and complex tree-based approaches) that you can encounter, within the setting of volatility forecasting (or forecasting in general).

  1. The mechanisms behind the results can be hard to explain: For forecasting purposes, there is a genuine "disapproval" of complex machine learning models, since you have a very limited understanding of "what is going on under the hood" and thus why many of these models are considered black-box algorithms. It is easier to interpret and understand what is going on in a GARCH model (this is just an example). Due to the transparency of the GARCH model, we can summarize the fallacies of said model and improve it, in order to incorporate more stylized features found in the market (eg. leverage effect). Complex machine learning models are, in general, hard to "improve" since you do not have the same transparency as with the GARCH model. Understanding your models weaknesses is also key to understanding when it will eventually break down.

  2. The running cost can increase substantially: Not only that, but in order to improve over the more classical models (eg. GARCH, HAR etc.) they need more data, which drives the cost up for companies that need new data-sources (which can be costly) only to get a marginal overall forecast improvement. This questions whether the marginal improvement (in any case) is worth the increased cost for licensing new data-sources.

  3. Parsimony?: When you're constructing a forecasting model (of any type) you want to adhere to the principle of parsimony: simple models with great explanatory and predictable power. This principle is not really satisfied whenever your machine learning model "functions" as a black-box algorithm.

There might be lesser complex models that circumvent all of the above defined worries and still have great forecasting performance. These are just things that are nice to be aware of.

Improve this answer
$\endgroup$
0
7
$\begingroup$

Was debating if I should even comment on this but then thought tonight I'm gonna have myself a real good time.

JPMorgan Machine Learning in Financial Markets Conference, Paris 2019 offers a superficial summary.

You simply do not have the data IMHO. To capture complex relationships you tend to have more parameters, which in turn leads to even more data requirements. The more time you spend with financial data, the more you realize it's remarkably noisy. On top of that, algorithms can only predict things consistent with what they have seen before. Not my answer - but here is an excellent summary.

Also, bear in mind, stock prediction is a very broad concept. Do you talk about someone who barely holds a stock for seconds? Or do you want to predict stock prices tomorrow, in 3 months or 10 years?

Seems most answers focus on the idea that someone with a profitable "model" would not want to share that with the public. Speaking of data, empirically, it seems to me that unicorns, planet 9 and primordial black holes and these folks have something in common. People (with unicorns it's mostly kids, with alpha less so, planets and black holes seem to fascinate adults and kids alike) tend to always talk about them, but no one has actually ever seen one for real.

Of course, there is Rentec, but generally, that also only applies to their Medallion Fund as their open funds frequently take a beating. Tradebot, and others, but there are not many who do something like this. This question goes hand in hand with the aforementioned summary which claims that many price changes are driven by news such as a plane crashing, terrorist attacks, virus outbreaks, tweets of Elon Musk or POTUS, merger announcements, and the like. Learning any pattern from these one-off events is likely more misleading than helpful.

Realistically, the outcome looks more like so. Spoiler alert, good luck for finding periods or strategies that outperform the S&P500. In the words of Warren Buffett, "I have talked to huge pension funds, and I have taken them through the math, and when I leave, they go out and hire a bunch of consultants and pay them a lot of money. Just unbelievable."

S&P Dow Jones Indices has good data. The Canadian example is an extreme outlier. As of Dec 31, 2020, 98.63% of funds underperformed the S&P/TSX Composite. However, US data does not look particularly good either. Pun intended, it’s bad and it’s getting worse. Maybe due to the emergence of ML? For readers who don't speak German, that article claims HIV may be connected to catalytic converters. Obviously complete rubbish but when you stare into the abyss, the abyss stares back (correlation does not imply causality; spurious, confounding, you name it).

It's difficult to find evidence against EMH and finding some "unicorns" is not sufficient evidence. I suspect they may be able to pull this off, without pure luck. Although statistically, the latter cannot be ruled out entirely, given the number of people who try to outsmart the average Joe. However, looking at these firms, they either employ technology to outrun competition (I know nothing about F1, but Bahrain last year showed the tools matter more than the craftsman) or some of the smartest people on earth, who do things intellectually, who I could not even dream about. Frequently, one goes hand in hand with the other. So to say the DC Universe of finance.

Nick Patterson gives a good overview about what they did at Rentec, which is mainly simple regression in his opinion (the whole podcast starts at 16:40, Rentec starts at 29:55 - a sentence before that is helpful).

This post has a good summary of some "basic" ideas to master for HFT. I think that is the only domain where you can consistently win but the users are like Lewis Hamilton. The best in their field, with the finest machinery on earth. Mainly, because you have a lot more data and the potential to exploit technological superiority. Compare FX tick data with +1 million quotes on any given day for liquid currency pairs vs 10 years of daily data. If you are in it for the long run, these ticks will be useless. If you have access to low latency of 20,000+ orders per second per single FIX, which is scalable to increase throughput and sub-millisecond roundtrip latency, the game will be different. InfoReach's website seems to be designed to Buffett's liking, no resources wasted to look pretty (although he would never need them).

The rest of us mortals lurks in the background and watches with disbelieve. Kenneth Rogoff and Richard Meese received an incredulous reaction to their now-famous paper showing that random-walk (RW) forecasts outperform economic models of exchange rates. Reactions were along the line of “You just cannot possibly have done it right” or "the results are obviously garbage". Rogoff makes an interesting point. After all, if money supplies are hard to predict, then one should not blame the models if exchange rates are hard to predict. Again unforeseen news that matters. Let's ignore that their finding was even more extreme. They tested predicting the exchange rate in one year, given the information about what money supplies, interest rates, and outputs are going to be in one year. However, even in this case, no economic model beat(s) the RW.

That paper was at a time when Warren Buffett frequently dwarfed the S&P500 returns. However, nowadays, the supply of capital chasing performance has dramatically expanded. Buffett no longer provides skyrocketing returns according to the usual first page of the annual letter to Berkshire shareholders comparing the annual performance of Berkshire against that of the S&P 500.

So to sum up, I think it may work for HFT, but the users are like Lewis Hamilton. The best in their field, with the finest machinery on earth. Overfitting noisy and irrelevant past data will not usually help much. I know many people will disagree, but empirical evidence tends to support my claims. Now is this all bad news? Not really. If an underlying follows a random walk, for example, Black Scholes and all its "derivatives" will likely be more useful. Probably also a reason why it is still used for pretty much everything (at least somewhere under the hood).

PS apologies for all the references, just had a real good time.

Improve this answer
$\endgroup$
5
$\begingroup$

Because it would not work. There was a very old paper several decades ago (dont recall but think it was in an IEEE journal on information theory) that showed that the stock market is RANDOM within an epsilon well smaller than the vig the brokers charge.

The sad truth is that all AI is genuine stupidity that assumes that correlation is causation. And those results are obfuscated by using huge fast complex programs manipulating large questionable piles of data that is too dirty and incomplete to be truly useful for such an application. Worse is that most assume a 'normal' distribution which is a logical fallacy. Unfortunately most people do not know enough about math and statistics to realize that some sort of beta distribution might be correct but that a normal distribution is always wrong.

That said, there are plenty of people who do write programs to predict stock results and some are widely sold, while others are sold to members of specific groups. But since none of them work right as shown by Taleb and his results noted in his book on Black Swans no reputable journal would hype them.

And as others noted above, if someone actually did have a method to predict stocks they would keep it totally secret and use it themselves.

Improve this answer
$\endgroup$
7
  • 3
    $\begingroup$ It depends on how you define "would work". There was a paper from 2011 that claimed that three day old tweets would yield a Sharpe of 9 on the Dow Jones index. It's been cited 5000+ times, secured tenure for it's author, and launched a cottage industry of sentiment data providers and soothsayers. I would say that 'worked' for some people. $\endgroup$
    – steveo'america
    Mar 17, 2021 at 21:29
  • $\begingroup$ I stand by the detailed article in the journal that I read that PROVED that the stock market is random within a very small epsilon. I also note that Taleb did well with Black Swans while the govt has had to bail out wall street and the banks when their methods totally failed. I also note that not everybody could use any method else there would be other problems. And finally I note that those who are 'successful' also have to be first to use the result. At least two groups have tried to find faster comms to send their orders in ahead of the others using similar methods that 'work'. $\endgroup$
    – statman
    Mar 18, 2021 at 0:18
  • $\begingroup$ Finally I note that works for some does not work as I would define it. If it would not work for me then I would not say it worked. You should feel the same way. Who was it who said they could sell ice to the eskimos? Like Barnum said, there is a sucker born every minute. And there is somebody out there with a system willing to fleece them with their stock trading system. $\endgroup$
    – statman
    Mar 18, 2021 at 0:25
  • 2
    $\begingroup$ @statman I don't see how it could possibly have been proven that it is random. Proven that it appears to be random based solely upon the data that the paper looked at, perhaps, but not more than that. E.g. if I give you the output of a high quality PRNG without mentioning that it is from a PRNG, you can show that the numbers appear to be random, but if you were to conclude that you've proven that it is random then you would be wrong because you lack the right data (the algorithm + the seed). $\endgroup$
    – JBentley
    Mar 19, 2021 at 8:21
  • 1
    $\begingroup$ @statman Claiming to prove that the stock market is random is equivalent to claiming that you have access to every single data source on the planet and that you've proven there is no correlation between temporally spaced events in any of those data sources. Doing that is simply not feasible. $\endgroup$
    – JBentley
    Mar 19, 2021 at 8:24
4
$\begingroup$

tl;dr– People don't publish trade-secrets.

They're trade-secrets.

Stuff like effective market-prediction algorithms tend to be trade-secrets:

Trade secrets are a type of intellectual property that comprise formulas, practices, processes, designs, instruments, patterns, or compilations of information that have inherent economic value because they are not generally known or readily ascertainable by others, and which the owner takes reasonable measures to keep secret.

—"Trade secret", Wikipedia [references omitted]

People don't tend to publish such trade-secrets for monetary reasons.

That said, academic publications often contain simpler work by both career-academics and rising professionals. When such research starts to look promising, the researchers may get offers to work in private, well-funded R&D labs where things are further developed. The original proof-of-concept research may stay in the literature while the trade-secrets are kept trade-secrets.

Improve this answer
$\endgroup$
2
  • $\begingroup$ On the sell side there would be incentive to share the research to juice volumes, though I would argue it's less likely to be valuable research. $\endgroup$
    – steveo'america
    Mar 17, 2021 at 17:38
  • 1
    $\begingroup$ These are literally "trade" secrets 🙂 $\endgroup$
    – arni
    May 5, 2021 at 20:35
4
$\begingroup$

I'm sceptical about the story of the "silent genius" who has a fantastic quant model, generating him more money than he can spend.

The Nobel prize is waiting for you

It is highly of interest of the academic world to explain (abnormal) returns. You could win the Nobelprize for discovering and explaining returns (see Fama-French). The publicity and the arising potential asset management mandates would make richer than your "quant strategy" can generate profits.

High standards for publishing in scientific papers

The overwhelming number of 'quant' strategies do not satisfy statistical tests and standards for testing or the theoretical explaination for the returns. But not being formally right doesn't mean that the strategy can't be profitable (at least for some time).

Improve this answer
$\endgroup$
3
$\begingroup$

I upvote @BobJansen's points; and add one small incremental observation.

Institutional investor scepticism about quant isn't so much scepticism about quant. It's more the fear that quant describes the current regime. Which investors believe (maybe correctly, maybe not) they already understand. Unless quant can tell them how this has changed, and do so fundamentally rather than quantamentally, then investors feel they have little need to learn from quant. So they are free to ignore...

Meanwhile, the peer-review process for academics acts as negative click-bait for institutional investors. Institutions worry to their wit's end about relative positioning risks, versus other institutions. So the benefits of any quant alternative have to be assessed against the risk that this exposes the insto to risks that other instos do not face, because they do not have the same quant!

The equal-and-opposite problem here is that the quants who can navigate investor-space have no incentive to publish. It's maybe not their hiding any "secret sauce". It's rather the reality of their jobs requires them to be ready to flip their views on a dime... which is hard in any peer-reviewed journal. "Sorry- the regime changed, so I changed my mind, what would you do different when the facts change, Madam? (Keynes)" isn't a script that financial journals appreciate... but it happens all-too-often on trading floors...

therein lies your problem, methinks. best, DEM

Improve this answer
$\endgroup$
2
$\begingroup$

A few year back a publisher tried to start a serious peer reviewed publication on machine learning. In a few hour over 6000 machine learning crowd signed a letter vowing to never publish or in any other way help any publisher in the current peer review system with machine learning. I signed it too, since the field is extremely fast moving and still the results are hard to explain. Machine learning is a field where gut feeling is still a good way to be better at machine learning. And then we have those publishers that have no clue what they are doing. They might publish, but the issue is that the publisher have problem to assets who is qualified to review a paper submitted. The academic consist to approx 99% of people who have made their degree in a field where machine learning could be useful and then as a doctor of some kind have started to be interested in machine learning. 99% because there is always a black swan out there.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.