# Historical Level 2 Data (Market Depth)

I'm currently looking to hone a system using market depth however I am looking for a good source of historical level 2 data.

As of right now the best source of historical I have found is automated trader, however at 6 grand for 1 year I need to make some more money before that happens. The other vendor I'm in talks with is dxfeed, however I'm not sure if they offer level 2 historical data.

For my platform I use DTN IQ feeding into matlab which sends trades to IB. DTN IQ does not provide historical level 2 data.

If I can't find a good source I'm instead going to test by paper trading the same system on the current market to hone in a strategy.

Any other thoughts or opinions on how to get access to historical level 2 would be much appreciated.

-
Just buy the historical data directly from the exchange. Probably not the answer you want, but it's the only one that's realistic. –  chrisaycock Mar 15 '12 at 15:22
For which asset class? If futures, take a look at CQG Data Factory . –  michaelv2 Mar 15 '12 at 17:05
I wonder how automatedtrader.net is able to sell native exchange data. Each exchange sells its own historical data and usually they are very restrictive about the distribution rights. You might want to compare those 3rd party prices against the exchange offer. –  Robert Kubrick Mar 15 '12 at 17:35
Exchanges sell through third parties. I have used trading physics which had more competitive prices. –  Steve Mar 17 '12 at 0:44
Dylan or anyione else How did he manage to connect IQFeeds with Matlab. I've ben looking all over the place for a Matlab toobox. As far as I'm aware Matlab's DataFeed toolbox only allows for Reuters, BLoomber, Yahoo and eSingals but not IQFeeds and they say will only be avaiable in Q4 2012 in version 2012b. Anyone know anything about this? thxs Jake –  user2643 Jul 3 '12 at 22:30

Collect the data yourself and self-timestamp it or buy it from a vendor that timestamps on receipt.

If you do buy data from the venues you should consider very strongly the fact that venue timestamps are never in agreement. Therefore, an event that INET reports at time A does not coincide with an event that BZX reports at time A. This makes buying this sort of data somewhat hazardous. You are better of collecting it yourself or buying it from a vendor who has collected it and self-timestamped the data on receipt.

-
Collecting the data yourself also means decoding the native exchange formats. Depending how many exchanges you need, this goes a long way from buying normalized data from vendors. –  Robert Kubrick Mar 19 '12 at 21:44
Of course it means writing feed handlers for each, that was implied. However, you can have accurate data or you can have crap from a vendor. You pick. Relying on venue specific time stamps will result in crap data. –  Louis Marascio Mar 19 '12 at 23:09

I use interactive brokers as well. What I do is I collect the data throughout the day with their API (reqMktDepth functionality) and I'm building my personal data warehouse. It may seem silly but I've been doing it for two years now and I have a good dataset to play with. Just get started and you'll see that it makes sense, the nice thing about it is that you only download whatever you really need and dont pay for the rest (actually you don't pay anything, period)

-
What if you want to investigate a new strategy and the data you've downloaded isn't sufficient? –  chrisaycock Mar 15 '12 at 16:27
The data you download from interactive brokers is standardized, you have a set of requests and you can only ask those requests, so in that sense the data I download is always sufficient (cause it's all there is, nothing more nothing less). What can be insufficient is which underlyings (i.e. contracts) you need for your strategy, they might not be in the set of contracts you have been downloading so far, that's true. –  mepuzza Mar 16 '12 at 8:54

dxFeed does offer level 2 historical data on demand. You can try it out for free. For example, to see how all market makers pulled out their quotes on ACN (Accenture) during May 6, 2010 Flash Crash go to the following URL: http://demo.dxfeed.com:7070/onDemand/data?categories=E-M&symbols=ACN&start=20100506-1447&end=20100506-1449 and use demo/demo when prompted to login. Add "&format=csv" at the end to download the data in Excel-friendly comma-separated values format. Demo access it limited to this particular day and to 1 second conflation interval.

Unless you host your own servers close to exchange and engage in nanosecond-level HFT, you don't have to build your own data warehouse. You can request the data you need from a vendor with adequate precision even for strategies on millisecond scale.

-

I agree with @Louis Marascio and @chrisaycock, data is expensive! Going directly to the exchanges is the safest way to obtain huge blocks of historical data, and real-time sources.

If $6000 for a year is too much, than your not going to be able to afford the exchange's data sources, than your not going to be able to afford a big chunk of historical data. Louis's point about generating your own database by either downloading it from your broker is a good one. Or consider subscribing to a more cost-effective service ($60-\$100 a month) and using that to build your database.

-