Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Hot answers tagged

57

NASDAQ makes this information available via FTP and they update it every night. Log into ftp.nasdaqtrader.com anonymously. Look in the directory SymbolDirectory. You'll notice two files: nasdaqlisted.txt and otherlisted.txt. These two files will give you the entire list of tradeable symbols, where they are listed, their name/description, and an indicator as ...


46

I am going to recommend something that I have no doubt will get people completely up in arms and probably get people to attack me. It happened in the past and I lost many points on StackOverflow as people downvoted my answer. I certainly hope people are more open minded in the quant forum. Note - It seems that this suggestion has created some strong ...


42

Consider the standard error, and in particular the distance between the upper and lower limits: \begin{equation} \Delta = (\bar{x} + SE \cdot \alpha) - (\bar{x} - SE \cdot \alpha) = 2 \cdot SE \cdot \alpha \end{equation} Using the formula for standard error, we can solve for sample size: \begin{equation} n = \left(\frac{2 \cdot s \cdot \alpha}{\Delta}\...


26

quandl is a new data source for all kind of econometric time series.


22

Don't use them. I have used them for years because I couldn't find another source that would provide all stocks in all US exchanges -- till now. But first, about eoddata: their data is very often missing elements, e.g., on any given day the SP500 index data my not be in their data set, even for a normal trading day their ftp files are often out of date, ...


21

Specialised NoSQL database systems are used a lot for time series storage, particularly for tick data: Kx / Kdb is one prominent solution; from Arthur Whitney et al who did A+ at Morgan Stanley Onetick is another, newer entrant, which has traces back to Goldman Sachs Voltdb is something by serial database inventer Michael Stonebreaker SciDB is another ...


20

Nanex has an interesting way of showing the order-book: The following images show CME's emMni future (S&P 500) depth of book and trades. The images are rainbow (ROYGBIV) color coded by the relative size at each depth level. Red indicates a lot of size, violet indicates size approaching 0. Note that a full minute before each event, the depth starts ...


20

Yahoo rounds the adjusted price to 2 decimals even though dividend amounts often have 3 decimal places. Since they apply the adjustment formula to adjusted prices, if you go far enough back in time, the value they give for Adjusted Price will be different than it would be if there were no rounding. edit: For example, for C (Citigroup), on January 2, 1990, ...


20

There is a very good reason why the ratio $$\frac {mean(HIGH-LOW)}{mean(|CLOSE-OPEN|)} \approx 2$$ on various financial series. If the price of a security evolves according to a Wiener process beginning at the opening bell and throughout the day, and the drift is negligible for that period of time, i.e.$\mu=0$, then the denominator of the above ratio ...


19

Bloomberg Open Symbology has this list. Look in the Common Stock precanned file. This will have a bit more data than you probably need as it has a separate entry and unique id for each place an equity is traded. However it is probably the highest quality list available for free anywhere. As for filtering ETFs are broken out in a separate file (Equity_ETP) ...


18

I don't know how interested you are in the CME data, but I have been learning about options and volatility modeling. I have been working with delayed CME data. I have been able to extract the JSON queries and now have been able to run them in my .NET application to get data for every asset type. Exmaple of ES options data: Run the query below in Chrome ...


16

Academic access to Thomson Reuters Tick History: www.sirca.org.au The Thomson Reuters Tick History database provides millisecond-timestamped tick data going back to January 1996, covering 45 million OTC and exchange-traded instruments worldwide. The database currently updates at a rate of 1 million messages per second and is around 3 Petabytes uncompressed....


15

Do not passively use Yahoo where you need reliable historical data; it will just fail at one point (from what I have seen due to corporate actions/dividends not properly implemented). Paying for a single alternative data source will not save you either (Bloomberg sometimes reports crazy intraday prices); the only way is to write some data cleaning routines (...


14

There are many specialised products for HF tick data. In addition to KDB which you mentioned, there is OneTick, Vertica, Infobright, and some open-source ones like MonetDB etc. (see http://en.wikipedia.org/wiki/Column-oriented_DBMS). My experience is that Column Oriented Databases are overrated when it comes to tick data, because very often you request the ...


14

That information is available in the SEC's EDGAR database, though there can be many flavors of "shares outstanding". It is reported quarterly in a company's 10-Q/K, sometimes on a weighted-average basis. If you don't want to get it manually, a service like Bloomberg will let you access the historical levels quite easily, or you can parse the EDGAR XML feeds. ...


13

The reason that "traditional" NoSql databases will not get much up take in finance is that they are designed to solve a different problem. Most NoSql databases from the web world are designed with two central design parameters. First key lookups should be very fast. Second is that operations should be atomic at the row level and should not span records. This ...


13

Your question is very vague (e.g. what are you trying to measure, and what "tick data" do you have), but I'll give you some pointers: In general, when people consider how prices evolve, they will tend to think about things like volatility and correlation dynamics. So I would start by defining exactly what you want to measure. The irregularity of time ...


13

Accounting is a vital skill if you end up in a managerial position, and unless your career goal is to always be a cog in someone else's clockwork, then you will eventually find yourself in a managerial/senior partnership position even through quant research. I still play a critical role in my firm's quant strategies team, but here's a few things I've had to ...


12

Fitch should be available right here: Sovereign Ratings History With Moody's it's not so easy, I don't know if there's a complete source available free of charge. But Sovereign Default and Recovery Rates, 1983-2007 has some data in the appendix III, though not so up to date and in a not that convenient format. Same goes about S&P, Sovereign Ratings ...


12

Google and Yahoo finance have a survivorship bias -- they only include firms that are still around. I know of no free source that provides the data you seek. I get my data from Compustat and CRSP via the Wharton Resource Data Service, but these (or Bloomberg or Reuters) are likely too expensive for an individual. Have you asked your broker if they will sell ...


12

There was a proxy called the ECU. You should be able to use the weights on the Wikipedia page to get a time series back to 1979. Alternatively, the St. Louis FRED also provides this time series.


12

Slightly ugly bash one-liner for a sorted JSON array: echo "[\"$(echo -n "$(echo -en "$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/nasdaqlisted.txt' | tail -n+2 | head -n-1 | perl -pe 's/ //g' | tr '|' ' ' | awk '{printf $1" "} {print $4}')\n$(curl -s --compressed 'ftp://ftp.nasdaqtrader.com/SymbolDirectory/otherlisted.txt' | tail -n+2 ...


12

From my experience, EODData is pretty much that you get what you pay for. Its not a very sophisticated product. They email you the files you subscribe to, and thats that. I have had an issue before of where the emails didn't go through anymore and I never heard anything from them. On Quality, I can't make a claim on its accuracy. It seems good. I find issue ...


12

Don't use them. Their data is very spotty. They offer "minute by minute" data on commodities markets and forex which should be technically 24hrs yet it usually starts from 9 or 11 o clock and the "minutes" end around 3 or 4. I emailed their support alias asking why this was the case. I also copied and pasted the opening and closing times of the exchanges and ...


12

What has changed The API is gone. The new downloads need 1st to parse the Yahoo Finance page to find a hidden crumb and use it as the key for data retrieval over a 2nd URL. The adjusted close is sometimes the adjusted, sometimes is the non-adjusted and sometimes is a different value Lines with literally "null" as the value for the prices have been ...


11

To get a consolidated feed of most of the data feeds here use Quandl. This is free for limited amount of requests per day.


11

I really wouldn't implement time series on my own unless I had a good reason to. AQR uses pandas, almost everyone in R using zoo or xts. I never like multiple parallel arrays, if it breaks everything is broken, plus it gets uglier as you increment data. If you are doing something in C++, why not have an array of structs for each object where you have ...


11

^GSPC is a price index, not a total return index, so it does not include dividends. SPY is an ETF that holds the underlying stocks. When it receives a dividend it keeps it in a cash account (which of course affects the NAV and market value of SPY shares) until the end of the quarter. At that time (on the 3d friday of Mar Jun Sep or Dec) it will pay out the ...


11

All of the answers above (unfortunately highly upvoted at this point) are missing the point. You shouldn't pick a DBMS or storage solution by general performance benchmarks, you should pick it by use case. If someone says they get a "x ms read", "y inserts per second", "k times speedup", "store n TB data" or "have m years of experience" and use that to ...


11

The indices have different quoting conventions. The way that a CDS index is traded is that you pay a fixed amount per year for protection in case of default (100 bps for IG, 500 bps for HY) and therefore the contract does not have a zero present value (as it would have if you paid the par spread, like in a fixed for floating interest rate swap). The amount ...


Only top voted, non community-wiki answers of a minimum length are eligible