In AI for algorithmic trading: 7 mistakes that could make me broke the author talks about why good forecasting cannot equal good trading based on the strategy selected. While reading the article I didn't quite understand his examples. Can anybody explain more?
3
-
3$\begingroup$ Could you make it more concrete what you did not understand? Otherwise I am afraid your question is not clear as it stands... Thank you $\endgroup$– vonjdFeb 11, 2019 at 19:02
-
1$\begingroup$ From a quick glance at the blog: I think the point is that if your input set is bad and your model performs good on this set, the actual trading will probably be bad. $\endgroup$– Bob Jansen ♦Feb 11, 2019 at 20:32
-
$\begingroup$ The article seems interesting but the pitfalls are not super clearly explained. Curious to see some detailed explanation too $\endgroup$– EzyFeb 12, 2019 at 3:16
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
The article probably could have used a bit of editing. Instead of talking about predictions, it should have talked about the impact of poor modeling decisions and data on predictions.
Most predictions do not consider the impact of adding one more actor to the results. For example, if you backtest a set of trades, then you are ignoring the fact that for your trade to be included for purchases, then it would have had to outbid all the trades that actually happened. Likewise, to sell, you would have to underbid all the trades that actually happened. Adding a player to the system changes the system. Had your algorithm been in the system and shifted both the supply and the demand curve, then definitionally your return would have to be less than or equal to the existing returns.
In addition, also not mentioned, is that if your algorithm becomes a regular feature of the market then the market-maker will adapt its inventory and liquidity position to your existence, at least for thinly traded securities. As with Long Term Capital Management, you could end up becoming the market and the illusion would be that your algorithm is working when in fact others are now playing off of it.
Except for small players, the illusion is that events are independent, except that is mathematically impossible as there is strong path dependence in this math. I have noticed that most discussions of such algorithms ignore the game theoretic elements and are straight regressions with no sense of price elasticity, liquidity costs, or the profit maximization and risk transfer strategies of the market makers.
I recently saw one of the worst algorithm design mechanisms that could be built. It attempted to maximize profit based on historical events. The historical profit-maximizing set of trades would have been purely random events. There would have been little parametric information to gather and even there it was captured wrong. The maximizing events would need to use extreme value theory not measures of central tendency, yet they used regressions looking for central tendency when they were looking at data sitting in the extrema. In doing this, they lost most hope of finding the drivers.
I think a lot of models are bad because many people are ignoring first principles, collecting weak data, and using cookbooks that may not apply to their problems.
The author is wrong about one thing though. If you had a perfect predictive tool that included the impact of your response, then you would make a lot of money. What the author is trying to convey is that people are not building those predictive algorithms.
$\begingroup$
$\endgroup$
Why Good forecasting != Good trading?
I am not yet familiar with the F1 score the author compares with the Sharpe ratio. But the article rightly points out at least two grounds on which good forecasting does not imply good trading.
The first issue has to do with the operative minutiae of trading. Models typically exclude aspects such as commissions, transaction costs, or any complications arising from market depth (I think this is what the author calls "bet sizing") once an order has been submitted. These operational hurdles can defeat the purpose of an otherwise profitable trade.
The second concern I perceived from the article is the "need [for] bids and asks from the order book", traders' focus being typically the HOLC (high, open, low, and close) prices. Incorporating & modeling bids and asks is certainly complex, but their due consideration can make the difference between an order being timely filled and being pointless (even if the model is systematically accurate about forecasting an average price).
