Was debating if I should even comment on this but then thought tonight I'm gonna have myself a real good time.
JPMorgan Machine Learning in Financial Markets Conference, Paris 2019 offers a superficial summary.
You simply do not have the data IMHO. To capture complex relationships you tend to have more parameters, which in turn leads to even more data requirements. The more time you spend with financial data, the more you realize it's remarkably noisy. On top of that, algorithms can only predict things consistent with what they have seen before. Not my answer - but here is an excellent summary.
Also, bear in mind, stock prediction is a very broad concept. Do you talk about someone who barely holds a stock for seconds? Or do you want to predict stock prices tomorrow, in 3 months or 10 years?
Seems most answers focus on the idea that someone with a profitable "model" would not want to share that with the public. Speaking of data, empirically, it seems to me that unicorns, planet 9 and primordial black holes and these folks have something in common. People (with unicorns it's mostly kids, with alpha less so, planets and black holes seem to fascinate adults and kids alike) tend to always talk about them, but no one has actually ever seen one for real.
Of course, there is Rentec, but generally, that also only applies to their Medallion Fund as their open funds frequently take a beating. Tradebot, and others, but there are not many who do something like this. This question goes hand in hand with the aforementioned summary which claims that many price changes are driven by news such as a plane crashing, terrorist attacks, virus outbreaks, tweets of Elon Musk or POTUS, merger announcements, and the like. Learning any pattern from these one-off events is likely more misleading than helpful.
Realistically, the outcome looks more like so. Spoiler alert, good luck for finding periods or strategies that outperform the S&P500. In the words of Warren Buffett, "I have talked to huge pension funds, and I have taken them through the math, and when I leave, they go out and hire a bunch of consultants and pay them a lot of money. Just unbelievable."
S&P Dow Jones Indices has good data. The Canadian example is an extreme outlier. As of Dec 31, 2020, 98.63% of funds underperformed the S&P/TSX Composite. However, US data does not look particularly good either. Pun intended, it’s bad and it’s getting worse. Maybe due to the emergence of ML? For readers who don't speak German, that article claims HIV may be connected to catalytic converters. Obviously complete rubbish but when you stare into the abyss, the abyss stares back (correlation does not imply causality; spurious, confounding, you name it).
It's difficult to find evidence against EMH and finding some "unicorns" is not sufficient evidence. I suspect they may be able to pull this off, without pure luck. Although statistically, the latter cannot be ruled out entirely, given the number of people who try to outsmart the average Joe. However, looking at these firms, they either employ technology to outrun competition (I know nothing about F1, but Bahrain last year showed the tools matter more than the craftsman) or some of the smartest people on earth, who do things intellectually, who I could not even dream about. Frequently, one goes hand in hand with the other. So to say the DC Universe of finance.
Nick Patterson gives a good overview about what they did at Rentec, which is mainly simple regression in his opinion (the whole podcast starts at 16:40, Rentec starts at 29:55 - a sentence before that is helpful).
This post has a good summary of some "basic" ideas to master for HFT. I think that is the only domain where you can consistently win but the users are like Lewis Hamilton. The best in their field, with the finest machinery on earth. Mainly, because you have a lot more data and the potential to exploit technological superiority. Compare FX tick data with +1 million quotes on any given day for liquid currency pairs vs 10 years of daily data. If you are in it for the long run, these ticks will be useless. If you have access to low latency of 20,000+ orders per second per single FIX, which is scalable to increase throughput and sub-millisecond roundtrip latency, the game will be different. InfoReach's website seems to be designed to Buffett's liking, no resources wasted to look pretty (although he would never need them).
The rest of us mortals lurks in the background and watches with disbelieve. Kenneth Rogoff and Richard Meese received an incredulous reaction to their now-famous paper showing that random-walk (RW) forecasts outperform economic models of exchange rates. Reactions were along the line of “You just cannot possibly have done it right” or "the results are obviously garbage". Rogoff makes an interesting point. After all, if money supplies are hard to predict, then one should not blame the models if exchange rates are hard to predict. Again unforeseen news that matters. Let's ignore that their finding was even more extreme. They tested predicting the exchange rate in one year, given the information about what money supplies, interest rates, and outputs are going to be in one year. However, even in this case, no economic model beat(s) the RW.
That paper was at a time when Warren Buffett frequently dwarfed the S&P500 returns. However, nowadays, the supply of capital chasing performance has dramatically expanded. Buffett no longer provides skyrocketing returns according to the usual first page of the annual letter to Berkshire shareholders comparing the annual performance of Berkshire against that of the S&P 500.
So to sum up, I think it may work for HFT, but the users are like Lewis Hamilton. The best in their field, with the finest machinery on earth. Overfitting noisy and irrelevant past data will not usually help much. I know many people will disagree, but empirical evidence tends to support my claims.
Now is this all bad news? Not really. If an underlying follows a random walk, for example, Black Scholes and all its "derivatives" will likely be more useful. Probably also a reason why it is still used for pretty much everything (at least somewhere under the hood).
PS apologies for all the references, just had a real good time.