Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange
47

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as people downvoted my answer. I certainly hope people are more open minded in the quant forum. Note - It seems that this suggestion has created some strong ...


24

You are not clear if you want the S&P500 index (SPY), OPRA Options or the Futures. Having spent a lot of time exploring vendors, here is a summary to help you (in alphabetical order): AlgoSeek.com : Intraday data back to 2007 for US Equities, Futures and Options. So you can get S&P 500 data. Intraday they have tick, 1 sec, 1 min and 5 min OHLC bars....


18

That was the devaluation of the French franc on 8 August, 1969 (by 12.5% in terms of par value), decided by President Pompidou and his finance minister Giscard d'Estaing. In those days, exchange rates were quasi-fixed, but subject to periodic realignments (the so-called Bretton Woods system of exchange rates, which was replaced by today's floating rate ...


12

A friend of mine has created a free source for fundamental financial company data of US listed stocks. All of the data is easily available from https://simfin.com/. Feel free to check it out and send him some feedback, so he can improve the service. Update 30.07.17: So he has just updated the site and now standardized company statements for over 2000 US ...


12

What has changed The API is gone. The new downloads need 1st to parse the Yahoo Finance page to find a hidden crumb and use it as the key for data retrieval over a 2nd URL. The adjusted close is sometimes the adjusted, sometimes is the non-adjusted and sometimes is a different value Lines with literally "null" as the value for the prices have been ...


11

The volatility in the indices long ago was similar in magnitude to what it is today. The problem you are seeing in your plots is one of compounding and scaling. Think of it this way- back in the mid 70's the magnitude of NASDAQ pricing was around \$100. Today it is on the order of \$4000, a change of 40x. In linear terms, a 1% change in the index today (\...


11

No - clearly you've not seen the licensing agreements the exchanges force you to sign (one way or the other). Generally such firms and individuals have greater utility from the money they'll make working with the data than risking going to jail. Market data is a 5bn / yr business. You're pushing the proverbial up-hill. Anyway, you can get financial index ...


11

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use case. If someone says they get a "x ms read", "y inserts per second", "k times speedup", "store n TB data" or "have m years of experience" and use that to ...


10

Non-disclosure agreements work on the legal side but not in reality, no agreement prevents someone with intent to still steal code or ideas. Protect core code in obfuscated code bases, through APIs installed on the local machine or have it on a server that others do not have access to to and provide access through function calls. Make sure the local machine ...


10

The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they store data along one of the dimensions (which you must choose, usually time) contiguously on disk, and thus reads are extremely fast. The alternative, when ...


9

The fx market, contrary to most other asset classes is an almost entirely fragmented over-the-counter market, aside the very small number of fx futures that are trading at dismal liquidity levels. Therefore, you will not encounter a single serious liquidity provider that will take a stab at estimating total traded volume in any of the currency pairs. Having ...


8

None of the previous answers have mentioned the fact that Bloomberg supports an API with support for all the main languages (C, C++, Java, Python, Perl -- and even Node and Haskell support on GitHub), on all the relevant operating systems: Windows, Linux, OS X, Solaris. This includes support for tick data which is stored in a rolling window (ie from ...


8

Markit Pricing Data is a prime source for cds data (not free).


8

I don't see anyplace obvious. My quick reading of Garber's original paper is that he greatly drew on work by Ernst Krelage which I've linked below. The Appendix I of Thompson (2007) describes in detail how he constructs his time series. I think you'll have to get your hands dirty, get into the weeds of these papers. This is probably a good thing, as the ...


7

You can get minutely as-traded prices for all US securities on Quantopian, for free. You can't download the original data, but you can query it, analyze it, and do your research within a hosted IPython notebook on the website. Once you register, go to https://www.quantopian.com/research and check the get_pricing() demo notebook. Disclosure: I work at ...


6

Both free and paid access to data sets conatianing company financial statement items is available from Quandl. The free data sets are sourced from the SEC based on compnay electronic filings and go back about five years. For example, you could obtain five years of MSFT's quarterly net income using the R call Quandl("RAYMOND/MSFT_NET_INCOME_Q") Lists of ...


6

You can find dataset you need on QUANDL (although some data service is for paying, it is a good data source and not expensive); there are a lot of fundamentals data, surely all you need and you download them in CSV format too. I also know that they provide different frequencies and so even the quarterly frequency. The only problem is that you have to ...


6

Let me guess, you fell for one of the fake Quantquote reviews and decided to purchase their buggy data? The reason for the missing quotes is Quantquote data is more of a snap-shot of market activity. It will not record every quote the way TickData or CQG does. ActiveTick is not as expansive as TickData but is more comprehensive than Quantquote. Maybe this ...


6

Dukascopy offers historical tick data. Through their historical data website you can download what you want, but registration is required, and lots of manual clicking. However if you are comfortable with scripting, you can directly download the tick data yourself. The URL pattern is http://www.dukascopy.com/datafeed/{currency}/{year}/{month}/{day}/{hour}...


6

To elaborate and emphasize a bit on what @Antoine says, using adjusted prices will be reasonable from a returns point of view, with dividends reinvested. That point, dividend reinvestment, is important because dividend reinvestment itself is a backtesting assumption, namely that dividends could be and would have been invested at the price you have in your ...


5

Depends on your budget of course, but: Mergent offers a great service, expensive though. Six Financial Information offers good Corporate Actions service. (Personally, I would go with this one) Morning Star Interactive Data (very hard company to deal with, their legal would waste a lot of your time, but may still worth to get a quote) Also, consider how ...


5

I think storing in UTC format is good practice. Here couple ideas that may motivate someone to deviate from that: Some markets are subject to day light saving time shifts and thus it introduces additional computations to convert back and forth, having to keep track of the 2 times a year the shifts occur. Some only limit themselves to an individual market, ...


5

Your prayers were heard ;-) The following article gives you all you need, especially the function getOptionQuote() which lets you download option chains for any ticker symbol with one line of code! You find the article here (with full R code): https://mktstk.wordpress.com/2014/12/29/start-trading-like-a-quant-download-option-chains-from-google-finance-in-r/...


5

As background, Floating point precision is a way of storing numbers such that the precision is relative to the largest digit. For instance, the number $0.00123$ stored in fixed precision needs 6 digits of precision (3 zeros and the 3 non-zero numbers). However, this same number stored as floating point precision $1.23 \cdot 10^{-3}$ needs only 3 ...


5

Thomson Reuters Tick History. Reasons you might want to use it: 1) It has just about everything you could possibly imagine, going back a long way, and more. 2) The API is (mostly) straightforward to use, so you can download tons of stuff without mucking around in Excel or spreadsheets of any kind. 3) The data can arrive pre-cleaned. Of course, it is ...


5

Compustat supports unlimited data export keeps the history of disbanded entities provides restatements since 1950 + point-in-time data since 1986 coverage since 1950 list of variables (data guide) Compustat is a S&P subsidiary. It goes as a plugin for CapitalIQ (also S&P), WRDS, CRSP, and other platforms. Pricing starts from \$3k. A platform ...


5

Morningstar Morningstar partnered with Quantopian, and the latter published the structure of Morningstar's equity fundamentals database: https://www.quantopian.com/help/fundamentals Quantopian users can use this data for free.


5

Better than Markit, you can have a look at https://www.datagrapple.com/ (subscription is free). About 1000 CDS are covered. Daily end-of-day prices (mid of a best bid/offer order book) from Jan 2006 and continues on an ongoing basis. There are the charts you want starting 2006. I think you may also be able to subscribe to an intra-day livefeed if you want.


5

I work at Quandl and I've listed some of our options databases below. Our API is free to use and data can be downloaded from our site in the tool of your choice, including Python - check out: https://www.quandl.com/tools/full-list Here are some of our options databases: Free: Chicago Board Options Exchange Premium: ORATS Option Volatility Surfaces US ...


5

In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of Quantitative Finance. The topic of P vs Q is presented in that dialogue in a manner meant to be understandable to a new comer. I would certainly recommend you to have ...


Only top voted, non community-wiki answers of a minimum length are eligible