12
$\begingroup$

I'm starting on a project that involves correlating and forecasting Forex time series to news releases. I'm aware of sources such as Thomson Reuter's machine readable news and Dow Jone's Newswire services. However, they both require expensive subscriptions. Does anyone know of any similar but free equivalent that I can use to develop a proof of concept?

I think the news releases do not have to be strictly real time, and some delay is fine as well but the timestamp is essential.


I think shouldn't have limited my question to just machine readable news which are preprocessed. What I'm interested in is more along the lines of Ransquawk's live headlines service which provides text-based news headlines updates. The text data aren't in a "machine readable" but that's fine as I can apply some simple NLP techniques on my side to extract information. The main issues is how to tap into these live feeds since most of them don't provide an API. And since what I'm working on is more a proof of concept, I think an archive timestamped news article can do as well.

$\endgroup$
  • $\begingroup$ Ransquawk has a JSON API $\endgroup$ – rtacconi Nov 27 '17 at 16:41
  • $\begingroup$ I believe there is a good JPM pdf named " Big Data and AI Strategies - Machine Learning and Alternative Data Approach to Investing " that circulate online and can be of interest to you. $\endgroup$ – were_cat Jan 10 at 13:09
5
$\begingroup$

It would be relatively trivial to implement a web scraper for any website you were interested in gathering news from - see Beautiful Soup for Python. This would allow you to gather and analyse news data from multiple sources in a way that may be more robust than relying on a single service. For example, you could screen scrape a certain website for the news headlines it's reporting, and then use a variety of statistical techniques to cluster and analyse the stories.

I'm not sure if this would fit with the terms of use of certain websites, so not going to comment on that.

$\endgroup$
  • 3
    $\begingroup$ The kind of machine-readable news used in trading is realtime, like weekly oil reserves as they are published. That kind of instant service definitely costs money, especially since the data is often routed directly to the data center that houses the exchange's matching engine. $\endgroup$ – chrisaycock Oct 27 '12 at 4:09
  • 3
    $\begingroup$ The original post stated "I think the news releases do not have to be strictly realtime and some delay is fine as well but the timestamp is essential. Thanks." Depends on the level of the delay I guess. $\endgroup$ – analystic Oct 27 '12 at 4:47
6
$\begingroup$

Concur with Thomas for most part, though I would recommend you to sign up for a trial with Dow Jones Newswire. I like the API and app that Newsware ( http://www.newsware.com/) makes available. It is not suitable for hft but I use it in order to stay informed and look up often used mnemonics. I think they have a pretty capable API and I remember they offer trials.

You may peruse it in order to calculate historical reactions to certain economic releases, however I am willing to bet you won't find ways to successfully (meaning, sustainable) forecast future price moves based on past news releases. I believe the only way to monetize price reaction to news and economic releases is through hft and employing a host of highly skilled linguists, programmers and economists. To my knowledge all software that is capable of analyzing text and deriving trading signals from news releases is stricly proprietary for the very reason that it costs a fortune to develop something that has an edge. Not exactly the playground for people on the retail side, and not even the playground for most investment banks, simply for lack of very specialized resources that are needed to be successful in this particular endeavor.

$\endgroup$
  • $\begingroup$ Is this API freely available? If not do you have any idea of the cost? $\endgroup$ – Datageek Mar 20 '14 at 16:51
  • 1
    $\begingroup$ yes it is, but please inquire with sales of Newsware regarding cost as I have replaced them with a different service. $\endgroup$ – Matt Mar 21 '14 at 10:05
  • $\begingroup$ Thanks Matt. Which service did you replace them with? Is the replacement better? $\endgroup$ – Datageek Mar 21 '14 at 12:41
4
$\begingroup$

I used https://newsapi.org/ for one of my last projects.

  • free
  • access to over 30,000 news sources world wide (US, Germany, India, Japan, etc.)
  • RESTful API returning JSON
  • excellent API documentation
  • no throttling

Example: Top Headlines

Request: https://newsapi.org/v2/top-headlines?country=us&apiKey=YOUR_API_KEY

Response:

{
  status: 'ok',
  totalResults: 36,
  articles: [
    {
      source: {
        id: 'cnbc',
        name: 'CNBC'
      },
      author: 'Lauren Thomas',
      title:
        'Target reports holiday same-store sales growth of 5.7%, maintains 2018 outlook - CNBC',
      description:
        "Target said its same-store sales climbed 5.7 percent during this past holiday season, topping growth of 3.4 percent a year ago and surpassing some analysts' expectations.",
      url:
        'https://www.cnbc.com/2019/01/09/target-reports-2018-holiday-sales.html',
      urlToImage:
        'https://fm.cnbc.com/applications/cnbc.com/resources/img/editorial/2017/12/04/104877704-RTX3JT8Z-2-shopping-target.1910x1000.jpg',
      publishedAt: '2019-01-10T11:30:10Z',
      content:
        'Target saw a surge of shoppers head to its stores and website this past holiday season, a sign that its investments in store remodels and delivery services are paying off, and an early sign that consumers across the U.S. spent more on gifts this year. div &gt… [+3929 chars]'
    },
    {
      source: {
        id: 'bloomberg',
        name: 'Bloomberg'
      },
      author: null,
      title:
        'Ford to Cut Thousands of Jobs in Europe, Eyes Plant Closures - Bloomberg',
      description: null,
      url:
        'https://www.bloomberg.com/news/articles/2019-01-10/ford-to-cut-thousands-of-jobs-in-europe-eyes-plant-closures',
      urlToImage: null,
      publishedAt: '2019-01-10T10:00:00Z',
      content:
        "To continue, please click the box below to let us know you're not a robot."
    },
    {
      source: {
        id: 'al-jazeera-english',
        name: 'Al Jazeera English'
      },
      author: 'Al Jazeera',
      title:
        'Turkey says will launch Syria attack if US delays troop pullout - Aljazeera.com',
      description:
        "Turkey and US remain at loggerheads over the future of Syrian Kurdish forces after Trump's decision to pull out troops.",
      url:
        'https://www.aljazeera.com/news/2019/01/turkey-launch-syria-attack-delays-troop-pullout-190110092123874.html',
      urlToImage:
        'https://www.aljazeera.com/mritems/Images/2018/12/16/fcef9a3bf7424348b7fe95c865939579_18.jpg',
      publishedAt: '2019-01-10T09:55:00Z',
      content:
        'Turkey will go ahead with an offensive against Syrian Kurdish fighters in Syria if the United States delays the withdrawal of its troops from the war-torn country, Turkish Foreign Minister Mevlut Cavusoglu has said.'
    }
    // ... and many more
  ]
};
$\endgroup$
3
$\begingroup$

News is not free, and hence you won't find a company offering machine readable news services for free. My best suggestion is to ask a machine readable news company for a day's worth of historical data. Even that might not work, however, as they won't waste their time if they don't think you're going to buy their service.

$\endgroup$
  • $\begingroup$ Examples of such companies? $\endgroup$ – Ryogi Oct 29 '12 at 18:59
  • $\begingroup$ @Ryogi bloomberg, thomas-reuters......... google is your friend $\endgroup$ – pyCthon Oct 30 '12 at 1:07
  • 2
    $\begingroup$ @pyCthon, yep, most answers to the questions on this site can be found on through google or google-scholar. But a good answer doesn't require you to sift through all that info. $\endgroup$ – Ryogi Oct 30 '12 at 6:30
2
$\begingroup$

If you are an academic interested in this field I would suggest contacting Sirca. Thomson Reuters is active with academics through their partnership with Sirca in Australia (www.sirca.org.au). Sirca has other machine readable text products available.

$\endgroup$
0
$\begingroup$

Cost-wise the following sources look promising. While not strictly finance related, you can filter the input news down to the categories applicable to your needs:

$\endgroup$
0
$\begingroup$

Free sources of machine-readable news abound in the form of RSS or Atom XML feeds. These formats are so popular that many of your favorite programming languages already have libraries that makes parsing them a plug-n-play task. Here are a couple of big time feeds to get you started:

  1. http://feeds.bbci.co.uk/news/rss.xml
  2. http://rss.msnbc.msn.com/id/3032091/device/rss/rss.xml
  3. http://rss.cnn.com/rss/cnn_topstories.rss

Feeds contain handy information such as title, time posted and often some summary of the content (first paragraph, first n characters, full content, etc.)

Feel free to contact me if you need further help with parsing. I recently put together an aggregator of over 200 feeds for data mining, quants, stats and open data, so this is still fresh on my mind: Quandl.io

$\endgroup$
-2
$\begingroup$

We have >20k global news sources in one API - multiple formats/delivery options. Everything is real time and up to micro for cross connections if you need algo-speed. We have the largest collection of global regulatory/disclosure wires in one pipe, plus all the leading heavy-weight financial newswires (rumors, broker ratings etc). All tagged with multiple time stamps, company identifiers, subjects, impact codes and more.

We have sizable archives too for back-testing.

Reach out to me for a free test feed and pricing. https://www.linkedin.com/in/carrie-farram-b443027/

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.