Is there any published research of decent quality linking news or unstructured information to asset returns? I know that Thomson Reuters offers its Machine Readable news (MRN), so somebody must use it. But I can't find much in the public domain.
I know that I have seen things like this in the past. Wasn't there something recently that used Twitter?
Here are a few recent papers as examples, although I will be brutally honest that I don't know if they speak to your decent quality requirement:
- "Trading Strategies to Exploit Blog and News Sentiment" (Zhang, Skiena 2010)
- "The Predictive Power of Financial Blogs" (Frisbee 2010)
- "An analysis of verbs in financial news articles and their impact on stock price" (Schumaker 2010)
Just FYI the Reuters product is called NewsScope.
The selling point is that they provide a sentiment reading per news item so the user doesn't have to do any NLP.
If you have a Reuters sales rep or contact them then they can get you several research/white papers that are interesting. Here are the ones I have been able to find online (my sales rep has provided me with better ones but I didn't save them):
Using their Event Indices product: http://puppetmastertrading.com/images/Reuters_NewsScope_Event_Indices_Whitepaper.pdf
Deutsche Bank's Quantitative Strategy (US) team put together the following piece on this topic (note: their research is available for clients, but I found that somebody uploaded the piece to a sketchy web site). In case the link dies, some of the academic papers they site are:
- Akbras, F., E. Kocatulum, and S. Sorescu, 2008, “Mispricing following public news: Overreaction for losers, underreaction for winners”
- Barber, B. and T. Odean, 2007, “All that glitters: The effect of attention and news on the buying behavior of individual and institutional investors”, Review of Financial Studies, Volume 21, Number 2
- Da, Z., J. Engelberg, and P. Gao, 2009, “In search of attention”
- Tetlock, P., 2009, “Does public news resolve asymmetric information?”
Their introduction says:
This month we tackle another new dataset: news sentiment. Regular readers of our research will know that this is a topic we find particularly interesting, and one that we have already done a lot of work in. In this particular report, we take what we think is an innovative approach to studying the predictive power of news sentiment; instead of using standard linear models, we focus on three non-linear, “learning” type models: classification and regression trees, forests of classification and regression trees, and multivariate adaptive regression splines. All three of these models are unique in that they allow us to take a datacentric approach to our analysis. Instead of predefining a hypothetical relationship and then testing it, we allow the data to determine the form of the model. This allows us to better understand which variables within our dataset are most important in determining post-event abnormal returns. It also allows us to model complex non-linear relationships that may not be apparent at first glance.
Overall we find that news sentiment, in conjunction with non-linear models, can generate alpha. Even better, we find this alpha is relatively uncorrelated with the more traditional quant factors. Of course, there is also a downside. The predictive ability of news sentiment is shortlived; the best results are obtained when forecasting only the next five days. Therefore, for some quantitative investors, the signal on its own may have too much turnover to be viable. Nonetheless, we do show that there are ways for even lower-frequency investors to use news sentiment data to enhance their stock-selection process.
A more recent DB Quant piece highlights another recent paper, Dzielinski, 2011, "News sensitivity and the cross-section of stock returns".
First of all, the author shows that there is, as expected, a statistical and economical difference in the returns on news days compared to non-news days. Also, while the direction of the difference is in accordance with the sentiment, the magnitude of the difference doesn’t relate to the news being positive or negative. These differences in returns between news and no-news days are actually heterogeneous among stocks: small and illiquid stocks tend to react more strongly, as do low book-to-market and high volatility stocks. From an industry point of view, the reactions also differ substantially, while still being significant, in each group. Interestingly, Dzielinski finally finds that there is a risk premium attached to news sensitivity, and that this phenomenon remains after controlling for well-known risk factors. The monthly return on the hedge portfolio is significantly different from zero and stands at 0.95% on average. The strategy still exhibits some significant loadings on some risk factors, as could have been expected from the panel regressions in sub-samples.
In the same piece, DB also mentions
A cautionary tale on all these approaches it told by Tim Loughran and Bill MacDonald in the Journal of Finance, 2011 (When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10-Ks, here).
In their analysis they show that the commonly used Harvard Psychosociological Dictionary is inadequate for sentiment classification in a financial context. Their findings are specific to the analysis of 10-k, but probably also indicative of the general difficulties with NLP in finance. Some findings:
- Most misclassifications simply introduce noise in the estimates;
- Some misclassifications introduce false positives (eg. "cancer" is normally negative, but in a financial context it is neutral, most likely it refers to an industry sector.
- A simple long-short strategy based on positive/negative words count yields small (positive) alphas which are not statistically significant.
There are of course several caveats:
- This approach is "mainstream" academic finance, with all its pros and cons (pros: clean approach, reproducible, simplicity suggests a low chance of data-snooping; cons: not strictly speaking quantitative, and - in this case - it doesn't use cutting the edge technology);
- The results are based on long horizon portfolio returns (buy/short and hold strategy on a 12-month horizon);
- The textual analysis is limited to low frequency information (10-Ks) as opposed to medium/high frequency information provided by news feeds.
A recent article by Frank Zhao is interesting to get started: Natural Language Processing - Part I: Primer.
You will find more papers on this repo (too long to copy all here): nlp_papers
If you are looking for possible applications of current SoTA research to financial markets, here is a quick list:
- Predict the impact of a particular report on the stock price.
- Predict the market capitalization based on the latest available quarterly reports, press releases, etc.
- Predict the future earnings growth based on the latest quarterly reports, press releases, conference calls, etc. available.
- Predict the credit rating (default probability) of a particular issuer given its reports (quarterly, press releases, etc). For example, it could be valuable to predict which bonds in your universe will go from BB to BBB (rising angels prediction) and vice versa.
- Predict the ESG scores of companies given their sustainability reports. It is hard to manually follow every company in your investible universe to assess their ESG scores (Environmental, Social, and Governance). NLP can help by digging into the companies documents.
- Predict the probability for a particular company to join the Norwegian blacklist (from the Norwegian pension fund).
- Predict the risk factors exposure of a stock given its quarterly reports and press releases. If a stock started trading only recently, you have very little information to assess its exposure to risk factors. NLP can help by using the reports of the company to predict its factor exposures.
- Predict the correlation/covariance matrix between assets. Useful if you do not have a significant historical period to compute the matrix.
Here is a project in Github https://github.com/cdubiel08/Earnings-Calls-NLP by some Rice University students that uses NLP to analyze earnings calls transcripts.
It gets the transcripts from Seeking Alpha web site, does some NLP, and tries to predict what will happen with the stock price.