50

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as people downvoted my answer. I certainly hope people are more open minded in the quant forum. Note - It seems that this suggestion has created some strong ...


24

You are not clear if you want the S&P500 index (SPY), OPRA Options or the Futures. Having spent a lot of time exploring vendors, here is a summary to help you (in alphabetical order): AlgoSeek.com : Intraday data back to 2007 for US Equities, Futures and Options. So you can get S&P 500 data. Intraday they have tick, 1 sec, 1 min and 5 min OHLC bars....


17

That was the devaluation of the French franc on 8 August, 1969 (by 12.5% in terms of par value), decided by President Pompidou and his finance minister Giscard d'Estaing. In those days, exchange rates were quasi-fixed, but subject to periodic realignments (the so-called Bretton Woods system of exchange rates, which was replaced by today's floating rate ...


14

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use case. If someone says they get a "x ms read", "y inserts per second", "k times speedup", "store n TB data" or "have m years of experience" and use that to ...


13

A friend of mine has created a free source for fundamental financial company data of US listed stocks. All of the data is easily available from https://simfin.com/. Feel free to check it out and send him some feedback, so he can improve the service. Update 30.07.17: So he has just updated the site and now standardized company statements for over 2000 US ...


12

The volatility in the indices long ago was similar in magnitude to what it is today. The problem you are seeing in your plots is one of compounding and scaling. Think of it this way- back in the mid 70's the magnitude of NASDAQ pricing was around \$100. Today it is on the order of \$4000, a change of 40x. In linear terms, a 1% change in the index today (\...


12

What has changed The API is gone. The new downloads need 1st to parse the Yahoo Finance page to find a hidden crumb and use it as the key for data retrieval over a 2nd URL. The adjusted close is sometimes the adjusted, sometimes is the non-adjusted and sometimes is a different value Lines with literally "null" as the value for the prices have been ...


11

No - clearly you've not seen the licensing agreements the exchanges force you to sign (one way or the other). Generally such firms and individuals have greater utility from the money they'll make working with the data than risking going to jail. Market data is a 5bn / yr business. You're pushing the proverbial up-hill. Anyway, you can get financial index ...


11

The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they store data along one of the dimensions (which you must choose, usually time) contiguously on disk, and thus reads are extremely fast. The alternative, when ...


10

Non-disclosure agreements work on the legal side but not in reality, no agreement prevents someone with intent to still steal code or ideas. Protect core code in obfuscated code bases, through APIs installed on the local machine or have it on a server that others do not have access to to and provide access through function calls. Make sure the local machine ...


9

The fx market, contrary to most other asset classes is an almost entirely fragmented over-the-counter market, aside the very small number of fx futures that are trading at dismal liquidity levels. Therefore, you will not encounter a single serious liquidity provider that will take a stab at estimating total traded volume in any of the currency pairs. Having ...


8

None of the previous answers have mentioned the fact that Bloomberg supports an API with support for all the main languages (C, C++, Java, Python, Perl -- and even Node and Haskell support on GitHub), on all the relevant operating systems: Windows, Linux, OS X, Solaris. This includes support for tick data which is stored in a rolling window (ie from ...


8

I don't see anyplace obvious. My quick reading of Garber's original paper is that he greatly drew on work by Ernst Krelage which I've linked below. The Appendix I of Thompson (2007) describes in detail how he constructs his time series. I think you'll have to get your hands dirty, get into the weeds of these papers. This is probably a good thing, as the ...


8

There is a deeper issue. Frequentist distributions are not probability distributions because they are designed to be minimax distributions rather than actual distributions. This ignores all of the other problems and this also ignores risk-neutral versus any other measure of risk aversion. An even deeper issue is that these models presume that the ...


7

You can get minutely as-traded prices for all US securities on Quantopian, for free. You can't download the original data, but you can query it, analyze it, and do your research within a hosted IPython notebook on the website. Once you register, go to https://www.quantopian.com/research and check the get_pricing() demo notebook. Disclosure: I work at ...


7

Let me guess, you fell for one of the fake Quantquote reviews and decided to purchase their buggy data? The reason for the missing quotes is Quantquote data is more of a snap-shot of market activity. It will not record every quote the way TickData or CQG does. ActiveTick is not as expansive as TickData but is more comprehensive than Quantquote. Maybe this ...


7

Markit Pricing Data is a prime source for cds data (not free).


6

Both free and paid access to data sets conatianing company financial statement items is available from Quandl. The free data sets are sourced from the SEC based on compnay electronic filings and go back about five years. For example, you could obtain five years of MSFT's quarterly net income using the R call Quandl("RAYMOND/MSFT_NET_INCOME_Q") Lists of ...


6

You can find dataset you need on QUANDL (although some data service is for paying, it is a good data source and not expensive); there are a lot of fundamentals data, surely all you need and you download them in CSV format too. I also know that they provide different frequencies and so even the quarterly frequency. The only problem is that you have to ...


6

Morningstar Morningstar partnered with Quantopian, and the latter published the structure of Morningstar's equity fundamentals database: https://www.quantopian.com/help/fundamentals Quantopian users can use this data for free.


6

Dukascopy offers historical tick data. Through their historical data website you can download what you want, but registration is required, and lots of manual clicking. However if you are comfortable with scripting, you can directly download the tick data yourself. The URL pattern is http://www.dukascopy.com/datafeed/{currency}/{year}/{month}/{day}/{hour}...


6

In their book "Counterparty Credit Risk, Collateral and Funding" D. Brigo, M. Morini and A. Pallavicini start with a dialogue between a Physics PhD graduate and an experienced practitioner of Quantitative Finance. The topic of P vs Q is presented in that dialogue in a manner meant to be understandable to a new comer. I would certainly recommend you to have ...


6

To elaborate and emphasize a bit on what @Antoine says, using adjusted prices will be reasonable from a returns point of view, with dividends reinvested. That point, dividend reinvestment, is important because dividend reinvestment itself is a backtesting assumption, namely that dividends could be and would have been invested at the price you have in your ...


5

I think storing in UTC format is good practice. Here couple ideas that may motivate someone to deviate from that: Some markets are subject to day light saving time shifts and thus it introduces additional computations to convert back and forth, having to keep track of the 2 times a year the shifts occur. Some only limit themselves to an individual market, ...


5

Depends on your budget of course, but: Mergent offers a great service, expensive though. Six Financial Information offers good Corporate Actions service. (Personally, I would go with this one) Morning Star Interactive Data (very hard company to deal with, their legal would waste a lot of your time, but may still worth to get a quote) Also, consider how ...


5

Your prayers were heard ;-) The following article gives you all you need, especially the function getOptionQuote() which lets you download option chains for any ticker symbol with one line of code! You find the article here (with full R code): https://mktstk.wordpress.com/2014/12/29/start-trading-like-a-quant-download-option-chains-from-google-finance-in-r/...


5

As background, Floating point precision is a way of storing numbers such that the precision is relative to the largest digit. For instance, the number $0.00123$ stored in fixed precision needs 6 digits of precision (3 zeros and the 3 non-zero numbers). However, this same number stored as floating point precision $1.23 \cdot 10^{-3}$ needs only 3 ...


5

Thomson Reuters Tick History. Reasons you might want to use it: 1) It has just about everything you could possibly imagine, going back a long way, and more. 2) The API is (mostly) straightforward to use, so you can download tons of stuff without mucking around in Excel or spreadsheets of any kind. 3) The data can arrive pre-cleaned. Of course, it is ...


5

This is an old post, but I thought I would offer the following facts: 1) QQ claims to be sited in the Empire State Building, Suite 2100. (https://quantquote.com/contact.php) That is false. They do not lease or sublease that address. 2) The VM message on their sales line is a standard carrier-provided one such as a private individual would have. It does ...


5

Better than Markit, you can have a look at https://www.datagrapple.com/ (subscription is free). About 1000 CDS are covered. Daily end-of-day prices (mid of a best bid/offer order book) from Jan 2006 and continues on an ongoing basis. There are the charts you want starting 2006. I think you may also be able to subscribe to an intra-day livefeed if you want.


Only top voted, non community-wiki answers of a minimum length are eligible