Hot answers tagged

67

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as people downvoted my answer. I certainly hope people are more open minded in the quant forum. Note - It seems that this suggestion has created some strong ...


27

quandl is a new data source for all kind of econometric time series.


23

Don't use them. I have used them for years because I couldn't find another source that would provide all stocks in all US exchanges -- till now. But first, about eoddata: their data is very often missing elements, e.g., on any given day the SP500 index data my not be in their data set, even for a normal trading day their ftp files are often out of date, ...


20

There is a very good reason why the ratio $$\frac {mean(HIGH-LOW)}{mean(|CLOSE-OPEN|)} \approx 2$$ on various financial series. If the price of a security evolves according to a Wiener process beginning at the opening bell and throughout the day, and the drift is negligible for that period of time, i.e.$\mu=0$, then the denominator of the above ratio ...


18

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use case. If someone says they get a "x ms read", "y inserts per second", "k times speedup", "store n TB data" or "have m years of experience" and use that to ...


17

Slightly ugly bash one-liner for a sorted JSON array: echo "[\"$(echo -n "$(echo -en "$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt' | tail -n+2 | head -n-1 | perl -pe 's/ //g' | tr '|' ' ' | awk '{printf $1" "} {print $4}')\n$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/otherlisted.txt' | tail -n+2 ...


16

Your question is very vague (e.g. what are you trying to measure, and what "tick data" do you have), but I'll give you some pointers: In general, when people consider how prices evolve, they will tend to think about things like volatility and correlation dynamics. So I would start by defining exactly what you want to measure. The irregularity of time ...


14

There are many specialised products for HF tick data. In addition to KDB which you mentioned, there is OneTick, Vertica, Infobright, and some open-source ones like MonetDB etc. (see http://en.wikipedia.org/wiki/Column-oriented_DBMS). My experience is that Column Oriented Databases are overrated when it comes to tick data, because very often you request the ...


14

From my experience, EODData is pretty much that you get what you pay for. Its not a very sophisticated product. They email you the files you subscribe to, and thats that. I have had an issue before of where the emails didn't go through anymore and I never heard anything from them. On Quality, I can't make a claim on its accuracy. It seems good. I find issue ...


14

That information is available in the SEC's EDGAR database, though there can be many flavors of "shares outstanding". It is reported quarterly in a company's 10-Q/K, sometimes on a weighted-average basis. If you don't want to get it manually, a service like Bloomberg will let you access the historical levels quite easily, or you can parse the EDGAR XML feeds. ...


14

Accounting is a vital skill if you end up in a managerial position, and unless your career goal is to always be a cog in someone else's clockwork, then you will eventually find yourself in a managerial/senior partnership position even through quant research. I still play a critical role in my firm's quant strategies team, but here's a few things I've had to ...


14

^GSPC is a price index, not a total return index, so it does not include dividends. SPY is an ETF that holds the underlying stocks. When it receives a dividend it keeps it in a cash account (which of course affects the NAV and market value of SPY shares) until the end of the quarter. At that time (on the 3d friday of Mar Jun Sep or Dec) it will pay out the ...


13

Don't use them. Their data is very spotty. They offer "minute by minute" data on commodities markets and forex which should be technically 24hrs yet it usually starts from 9 or 11 o clock and the "minutes" end around 3 or 4. I emailed their support alias asking why this was the case. I also copied and pasted the opening and closing times of the exchanges and ...


12

The standard answer is going to be that for time series, you want a column store database. These are optimized for range queries (ie: give me everything between two timestamps) because crucially, they store data along one of the dimensions (which you must choose, usually time) contiguously on disk, and thus reads are extremely fast. The alternative, when ...


12

What has changed The API is gone. The new downloads need 1st to parse the Yahoo Finance page to find a hidden crumb and use it as the key for data retrieval over a 2nd URL. The adjusted close is sometimes the adjusted, sometimes is the non-adjusted and sometimes is a different value Lines with literally "null" as the value for the prices have been ...


11

To get a consolidated feed of most of the data feeds here use Quandl. This is free for limited amount of requests per day.


11

The indices have different quoting conventions. The way that a CDS index is traded is that you pay a fixed amount per year for protection in case of default (100 bps for IG, 500 bps for HY) and therefore the contract does not have a zero present value (as it would have if you paid the par spread, like in a fixed for floating interest rate swap). The amount ...


10

Somewhat more economic data can be found at e.g.: The World Bank The United Nations The OECD More financial: The IMF European Union / EFTA / EMU data: Eurostat European Central Bank (financial) Data from these sources is all freely available. You can also play with data from many of these sources using the Google Public Data Explorer.


10

Quandl is a free one, with good economic and market data and an API http://www.quandl.com/


10

Non-disclosure agreements work on the legal side but not in reality, no agreement prevents someone with intent to still steal code or ideas. Protect core code in obfuscated code bases, through APIs installed on the local machine or have it on a server that others do not have access to to and provide access through function calls. Make sure the local machine ...


10

There are so many different data providers, and they all end up using slightly different definitions. For Google, it looks like they use Deutsche Börse (Google) as a data source. I can't tell what Yahoo Deutschland is using. I think your real question, though, is why the different data providers have different answers to the same questions. The answers ...


10

As a complement to the answer by Enrico, the ISIN identifies a financial instrument in a unique way. Tradition is, of course a reason and many codes, such as the CUSIP code in the US, predate the ISIN. But there are a few other solid underlying principles that are applied in financial industry. These are pieces of information that are not necessarily "...


9

Although quite simple connected scatterplots can give interesting new insights on how time series perform together: http://steveharoz.com/research/connected_scatterplot/ As an example: Gold vs. S&P 500 from 1970 till today: The green point marks 1970, the red point is today. Every point is a year, moving vertically upwards means rise in the S&P ...


9

Don't use eoddata unless you absolutely have to. I purchased long history of global index and forex prices. There are hundreds and hundreds of incorrectly missing or completely wrong prices. For example (big problem with forex), the price is coming along in the 10s, then spkes up to the 1000's for a few days before dropping back down. Repeat over and ...


8

And then music... Victor Neiderhoffer, in a 2001 interview: The market plays music all the time. The problem is you never know how the music of the market is going to end. But a good framework is that it will end on the tonic. Consonance to dissonance back to consonance. And whenever there's tremendous dissonance, strident moves in one direction, a good ...


8

I am an Oracle Database Administrator. I subscribe to EODdata for daily minute-tick price data. When I upload their datasets into my database I run consistency checks before the data is finalized. I can confirm the wildly swinging price spikes other users have identified, plus data gaps, plus the same stock listed on multiple exchanges, plus entire ...


8

It seems logical to me to have a Financial accounting course in a quant program. Quants can have a lot of different occupations, from derivative pricing to quant analyst in a "research" (i.e. analysis) dept. of a broker, a risk dept., a fund (as an analyst or as a potfolio manager), or quant execution trader (the list is far longer). In the case of being ...


8

None of the previous answers have mentioned the fact that Bloomberg supports an API with support for all the main languages (C, C++, Java, Python, Perl -- and even Node and Haskell support on GitHub), on all the relevant operating systems: Windows, Linux, OS X, Solaris. This includes support for tick data which is stored in a rolling window (ie from ...


7

You can consider the following two methods: If you have Matlab, you can use the following code: Volume Weighted Average Price from Intra-Daily Data This package allows you to [1] retrieve intra-daily stock price data from Google Finance, [2] calculate the VWAP at the end of each trading day and [3] transform intra-daily data to a daily format. If you ...


7

To me, coloring by data value is a great way to bring applications alive. If traditional ways are not enough, probably taking 3D in use would be a way: And of course 2D heatmap is a very handy for sure. I'm developing data visualization software components with 3D technologies, so definitely all feedback and ideas are welcome :-)


Only top voted, non community-wiki answers of a minimum length are eligible