Questions about handling, obtaining, generating, or analyzing all types of financial or economic data.
93
votes
19answers
24k views
What data sources are available online?
What sources of financial and economic data are available online? Which ones are free or cheap? What has your experience been like with these data sources?
36
votes
7answers
5k views
Innovative ways of visualizing financial data
Finance is drowning in a deluge of data. Humans are not very good at comprehending large amounts of data. One way out may be visualization.
Traditional ways of visualizing patterns, complexities and ...
25
votes
6answers
4k views
What broker/feed/APIsetup allows for recording the most accurate data (cheaply)?
I'm currently using IB's Java API and getting feeds through them. However the real-time feed is updated only every 250ms and the historical feed only every second.
I'm primarily looking for ES data ...
18
votes
5answers
3k views
What is a good broker for HFT?
Currently I trade trough IB. I run my HFT strategies (100 roundtrips per hour) but I think that latency is killing me and my profits are shrinking. I need the fastest possible brokers out there which ...
17
votes
7answers
6k views
How should I store tick data?
How should I store tick data? For example, if I have an IB trading account, how should I download and store the tick data directly to my computer? Which software should I use?
17
votes
2answers
1k views
How much data is needed to validate a short-horizon trading strategy?
Suppose one has an idea for a short-horizon trading strategy, which we will define as having an average holding period of under 1 week and a required latency between signal calculation and execution ...
16
votes
2answers
2k views
Do you have historical tick data you want to donate?
Do you have historical market/pricing ticket data that you would like to donate to the Open Source Trader project (OST)??
Please: upload your files! Once we gather some data, we'll do our best to ...
15
votes
8answers
2k views
What kind of basic framework or application do you use to run your trading algorithms?
I heard about MetaTrader from http://www.metaquotes.net. Is there any other framework or program available? Do you use different software for backtracking and running your trading algorithms?
Thank ...
14
votes
5answers
3k views
Training set of tick-by-tick data?
I'm looking to find a free source of tick by tick data (<1sec) for training purposes. It doesn't need to be longer than a day, and I don't care what instrument, or exchange, or time it is. I just ...
14
votes
7answers
4k views
Any known bugs with Yahoo Finance adjusted close data ?
Yahoo Finance allows you to download tables of their daily historical stock price data.
The data includes an adjusted closing price that I thought I might use to calculate daily log returns as a ...
13
votes
4answers
4k views
Usage of NoSQL storage in Finance
I am wondering if anyone has used NoSQL (mongodb, cassandra, etc.) to store and analyze data.
I tried searching the web but was not able to see if the financial firms had gotten in to using nosql ...
13
votes
3answers
3k views
Free market data (delayed or snapshot)
I know there are similar questions but I don't think there are any identical ones.
Basically, I'm looking for one of these two things.
Delayed market data feed. A tick by tick feed, but delayed by ...
12
votes
6answers
2k views
What is the best data structure/implementation for representing a time series?
I was wondering what is best practice for representing elements in a time series, especially with large amounts of data. The focus/context is in a back testing engine and comparing multiple series.
...
12
votes
1answer
364 views
What time are Bloomberg Open Symbology Files updated daily?
Does anyone know what time the Bloomberg Open Symbology precanned files are updated every day? I am planning on upating my view of them at 9 AM EST every morning. Furthermore, will the files for ...
11
votes
3answers
3k views
R: How feasible is it to store — and work with — tick data in a database connected to R?
I'm looking to convert some tickdata .csv files into a database on a local disk and then use R to call the data and do my various analytics and modelling.
What are some best practices / ...
11
votes
4answers
2k views
data on historical stock price of bankrupt companies
does anybody know a site where I can download historical data on stocks including companies that have gone bankrupt such as lehman brothers?
it appears that bankrupt companies no longer appear in the ...
11
votes
2answers
898 views
How to update an exponential moving average with missing values?
Say you have an Exponential Moving Average being continuously updated over a time series using 1-second-long time periods. What should happen if there is no value for the next second, e.g. there were ...
11
votes
3answers
1k views
Free intra-day equity data source
Are there any free data source for historical US equity data? Yahoo finance has daily prices but I'm looking for something more granular and goes back 2 or more years (doesn't have to be close to tick ...
10
votes
2answers
551 views
Why is the ratio of Hi-Low range to Open-Close range close to 2?
I tried it in several symbols and timeframes with the same result:
$$\frac {mean(HIGH-LOW)}{mean(|CLOSE-OPEN|)}$$
...
9
votes
2answers
203 views
Effective Euro-USD (EURUSD) Exchange Rate Prior to Euro's Existence
Motivation: I am running a quantitative analysis that requires long-term, exchange rate data.
Problem: Does anyone have methods for dealing with the EURUSD exchange rate prior to the Euro's ...
9
votes
3answers
400 views
Literature on generating synthetic time series for testing
I have some market data (daily time series) for bond prices and CDS indices and I would like to generate synthetic versions of these which are statistically "similar" for testing trading strategies. ...
9
votes
2answers
312 views
How to “uncluster” a set of financial data?
I am attempting to evaluate and compare the profit factor of different "test runs" of a FOREX trading strategy.
My problem is that, despite an average time between orders of 2hr+, some of these runs ...
9
votes
1answer
232 views
Is there a standard / methodology to determine and grade the quality of OHLC data?
Inputs are most important to any decision making.
For strategy backtesting, the OHLC data is one of the most inputs.
So to ensure the correctness and integrity of OHLC data, we have checks on the ...
8
votes
7answers
2k views
Is “eoddata” a good data source?
Not sure if this is a relevant question for site, but I am looking to move to www.eoddata.com as my data source.
If anyone has used it, can you tell me how the data quality is ?
I am currently ...
8
votes
4answers
341 views
Seeking Historical Non-Finance Datapoints for Backtesting
I'm working on some financial analysis code which I'd like to test against a historical dataseries to analyze the correlations to my algorithm to some non-finance related data. Ideally, I'd like to ...
8
votes
3answers
601 views
Why isn't all market data free?
I am hoping any answers can be threaded somewhere between "don't be naive" and starting a quasi-political flame.
Transparency is suppose to be good for markets and also a social good. The market ...
8
votes
3answers
842 views
How to generate synthetic FX data for backtesting?
I want to generate synthetic forex data for the purpose of backtesting my trading algorithms. I have some rough ideas in mind on how to do this:
Start with a curve representing a trend, then randomly ...
8
votes
3answers
1k views
Earnings and valuation data sources online
Are there any free/cheap sources for historical data on company earnings and valuations? I can get historical price data from Google and Yahoo, and it looks like I can get about five years of ...
8
votes
1answer
294 views
Where can I find data on the interbank lending market?
Where can I find disaggregated interbank lending data (i.e. bank A lends to bank B x money at y rate)? I could only find data on interest rates.
I would accept LIBOR market data as well as any ...
8
votes
2answers
683 views
time series management system
I'm happy how we store a single time series but we somehow lack a system that glues them all together. I'm talking about a few million time series coming from ~50 data vendors and representing maybe ...
7
votes
4answers
1k views
Real-time & Fast S&P 500 E-Mini Futures (ES) Data
I trade on local exchanges in Europe, but my HFT strategies need S&P 500 e-mini futures data (ES). I don't need to trade ES, but I need real-time data and I want to have it as fast as possible.
...
7
votes
3answers
298 views
Is there data on market participants at a particular moment?
I am looking for data on market participants at a particular moment (or some proxy/approximation). For example, how can I tell whether mostly big players and HFTs are dominating the market in ...
7
votes
2answers
954 views
Is there a technique for using xts or zoo objects with options data (i.e., many entries per date) in R?
I am starting to work with options data from optionmetrics. I use data frames, but it seems like xts or zoo objects are the way to go for features and speed. I can't figure out the best work-around to ...
6
votes
4answers
6k views
Where to download list of all common stocks traded on NYSE, NASDAQ and AMEX?
I have a very basic data question: how to get a list of all common stocks traded on NYSE, NASDAQ and AMEX? I would need to be able to get the approximate list of common stocks as is available in ...
6
votes
4answers
349 views
Best way to store hourly/daily options data for research purposes
There are quite a few discussions here about storage, but I can't find quite what I'm looking for.
I'm in need to design a database to store (mostly) option data (strikes, premiums bid / ask, etc.). ...
6
votes
4answers
213 views
What is a sound way to project Company X's earnings over the next Y years?
I need to estimate cumulative earnings over the next Y years and I'd like to find a solution that is theoretically sound and relatively simple. Can anyone recommend an approach?
Given:
I have 30 ...
6
votes
3answers
817 views
Which data service to buy for redistributable data?
This is a follow-up to my previous question regarding anyone whom wanted to 'donate' data.
So far no one has stepped up (learning more about buying and selling data, I realize that I'm going to have ...
6
votes
2answers
817 views
Non-SQL methods for high-frequency accounting?
Does anyone know of any prior art for non-SQL data structures for high-frequency accounting, whether client, broker, or exchange-side? I'm thinking specifically of the problem of booking individual ...
6
votes
2answers
209 views
Where can you find data on non-trading stocks?
My data source for end of day prices only gives the prices of the trades during the day. If a stock stops trading/goes out/moves to another exchange|changes symbol. I never hear about this.
My ...
6
votes
1answer
270 views
NASDAQ TotalView ITCH order reference number number characteristics
I am building a custom hash implementation for storing NASDAQ ITCH order messages. Obviously this is keyed on the order reference number and I am wondering if these numbers are sequential, random or ...
5
votes
4answers
483 views
Free data on swap options
I am trying to analyze valuation methods for swaptions. Does anyone know of free example data for these OTC-traded securities?
5
votes
2answers
547 views
FX Tick Data question
Anyone can tell me what the first column and the last column in this FX tick dataset mean? The first seems like some kind of ID, and what D in the last column mean?
368412956 AUD/CAD 12/30/2007 ...
5
votes
4answers
864 views
Analyzing tick data
What are some of the commonly used techniques to analyze tick data?
I am looking at tick data to see how the quotes/ mid-price evolves due to certain events in the market. Since tick data is ...
5
votes
1answer
387 views
Automated 10-K XBRL data grab using the SEC file structure
I would like to write a program that takes as input a list of CIK/year/quarter entries. The program should iterate through the list and, for each entry, grab XBRL financial data from the SEC website ...
5
votes
1answer
315 views
Historical Hedge Fund Index Data
Can anyone point me to some Hedge Fund index data - daily levels of the HFRX or something similar, that is available for free and has history back to 2007? The data available through my broker seems ...
5
votes
1answer
660 views
A generic limit order book: What are the most important queries it should be able to answer?
Assume a class LimitOrderBook which represents a limit order book in a trading system.
To be able to represent the limit order book a data handler reads a feed ...
5
votes
2answers
513 views
Free/cheap source of structured historical quarterly filings?
What is a good place to acquire cheap/free historical quarterly filings data for US Companies? Specifically, I'm interested in the consolidated financial statements. The data should be well-structured ...
5
votes
1answer
153 views
Reliable Economic Data on China
Trying to develop a new strategy, but I need to find reliable economic data on China. It's well known that the official figures there don't tell you everything (to put it nicely), so I was wondering ...
5
votes
1answer
131 views
Availability of machine-readable OCC Infomemos?
Does anyone know of a source of machine-readable (XML,etc) OCC Infomemos? The PDF files available below contain all the information I want, but are a pain to parse.
...
5
votes
0answers
200 views
option chain data visualization, sunburst
I think option chains are not represented in the best way. With more and more options products coming out and trading on the various exchanges, I see vendors struggling to keep up with a good way to ...
