# Tag Info

13

There are many specialised products for HF tick data. In addition to KDB which you mentioned, there is OneTick, Vertica, Infobright, and some open-source ones like MonetDB etc. (see http://en.wikipedia.org/wiki/Column-oriented_DBMS). My experience is that Column Oriented Databases are overrated when it comes to tick data, because very often you request the ...

8

The main issue measuring intraday volatility is called "signature plot": when you zoom in, the volatility measure (i.e. empirical quadratic variations) explode. Similarly you have the "Epps effect" for correlations: when you zoom in, the correlations collapse (it is at least a mechanical effect). For the volatility a lot of models can correct this: - first ...

8

Here are some pointers. First of all: What you list as a Reuters RIC, RSF.ANY.AAPL.OQ, is not really a RIC, only the AAPL.OQ is. The initial part is some stuff which is essentially site specific and tells me that you are working on a site that has a legacy RTIC infrastructure (some Reuters/TIBCO technology which is quite old these days and for all ...

7

Your question is very vague (e.g. what are you trying to measure, and what "tick data" do you have), but I'll give you some pointers: In general, when people consider how prices evolve, they will tend to think about things like volatility and correlation dynamics. So I would start by defining exactly what you want to measure. The irregularity of time ...

6

The expression you have is fine. But more generally, for the intraday volatility, I don't think there "the correct definition". More like, whatever works in the given context. I found the following notes by Almgren pretty useful: http://cims.nyu.edu/~almgren/timeseries/notes7.pdf

6

We use Node for reporting but not as part of our main signal generating trading system. To be honest the answer will almost certainly be yes for every common programming technology as it just takes one person to use it somewhere to make the answer yes. Just look at OCaml, before Jane street, most techno logiest on the street had never heard of it and now ...

5

If you're missing ticks, then no technique will get those ticks back. If you have two sources, then designate one source as the primary feed and then fill-in gaps from the secondary feed. Of course, you'll have to mind the timestamps when determining whether the secondary feed can be used properly.

5

Yes, there is a software application that you can purchase for $39.99 which stores all your tick data in a highly compressed format while still allowing maximum throughput and lowest latency data queries that I have ever seen. The package provides APIs to all languages under the sun but because they have a special sale going on it comes with the complete ... 4 Statistical volatility is the standard deviation of a window of log returns. For example, 30-day statistical volatility is the standard deviation of 30, one-day log returns. The log return comes from the assumption that log stock returns are normally distributed. Statistical volatility differs from implied volatility which is the volatility input to some ... 4 xignite seems like a cheaper option for historical data at around$6,000 per year. I am not a client (yet) so I can't give my opinion but it looks good to me. They have a long list of services, a free trial period and interesting pricing model based on the number of queries you send them. If you do use them, let us all know.

4

I think you're addressing the wrong people. What I'd do is to talk to the data vendors' marketing departments, much like I'm sure http://www.wikiposit.org/ has done. Tell them you would like to offer (an excerpt of) their data, in a unified/post-processed form, you would be willing to include links back to their sales website, or praise the vendors ...

4

Collect the data yourself and self-timestamp it or buy it from a vendor that timestamps on receipt. If you do buy data from the venues you should consider very strongly the fact that venue timestamps are never in agreement. Therefore, an event that INET reports at time A does not coincide with an event that BZX reports at time A. This makes buying this sort ...

4

Using MySQL for financial data is not unreasonable. But for tick data are you ever going to do anything except a query on a date range? For analyzing tick data in R I generally keep it in a disk file, one tick file per day, and load the files in as I need them. Using .RData files instead of csv files is quicker. I've also used custom C++ classes before, to ...

4

As explained in the comments best bid and best offer (best ask) are the best prices at which you can respectively sell and buy at least one unit of the asset your are considering. When backtesting a strategy, most people usually either use best bid and best offer or even worse last price. The problem is that these prices are only available for a limited ...

3

If you just want to run some simplistic technical analysis on quotes, then select the last quote for each unique timestamp. That will ensure that you don't have duplicate timestamps. If you must have it evenly spaced (i.e. no gaps from one second to another), then you can reuse the previous quote to fill-in the missing value.

3

Bloomberg equity codes are usually quite easy to derive if you know the ticker (though you may have to replace dashes/spaces/slashes for preferred and multi-class shares). This is just for equities though and this is definitely not the case for futures. RICs are a different story. In the US, you need to know on what exchange the ticker is listed to get the ...

3

Obviously merging two streams is harmless and it should be done. But it's hard to advise you regarding the "interpolation" methods you can use to generate the ticks without knowing why you need this. The reason is that any method will introduce a certain bias to the data. Therefore, it very much depends on what are you going to do with your altered data on ...

3

I use interactive brokers as well. What I do is I collect the data throughout the day with their API (reqMktDepth functionality) and I'm building my personal data warehouse. It may seem silly but I've been doing it for two years now and I have a good dataset to play with. Just get started and you'll see that it makes sense, the nice thing about it is that ...

3

On a theoretical level and for low frequency data (e.g. daily), your formula seems right. However, since you are talking about one minute bars, things may get a little messy. There is a vast literature on this, and empirically, things are complicated due to market micro-structure noise. Namely, you need to do consider jumps, errors, periods of low volume, ...

3

Your code for volatility seems correct, if you want minute volatility, but is that really what you want? See this recent question on annualizing volatility from intraday data. Also, using first and last tick is what is generally done, but over very short time intervals such as a minute, you will have microstructure issues. Another question here deals with ...

3

I'm trying to formulate an answer for latter users of the site and, hopefully to lure some more experienced high frequency traders to further react to my post. First of all, I'd notice that the paper you mentioned was published in 1991. I believe high-frequency trading has evolved a lot since then, and that nowadays HF strategies are more evolved. From ...

2

It isn't clear from this post or your last post what data you want and what you want it for. The only thing you are going to get permission to distribute is the higher level data feeds like EOD data, or last sale (probably can't offer history). Exchanges typically have this information available on their websites, for instance NASDAQ Basic pricing. If you ...

2

Following on from Tal Fishman's idea of using "some good old-fashioned judgment," you might find the idea of applying a Tukey chart and its related upper and lower control limits more useful than standard deviation.

2

This sounds like a case where you will need to apply some good old-fashioned judgment to determine what the standard deviation "should be" before you have enough data to measure it. Surely this process repeats with some frequency, and perhaps given some attributes and more details you could make an educated guess as to the standard deviation (or quantiles, ...

2

I will be adding here some resources that I found during the research, hopefully it will turn useful for others: http://www.linkedin.com/groups/master-symbol-database-most-cost-98173.S.51776671 http://en.wikipedia.org/wiki/Reuters_Instrument_Code

2

I have a little experience with this. First, NASDAQ has shared a dataset with researchers that flags whether an HFT participated in each trade or not but not the actual MPID - probably less granular than what you want. You generally need a professor to "cosign" your request, write a brief project proposal, and sign an NDA to get it. They also have shared ...

2

Neither is entirely correct. I think you need to find a middle ground here because: a) Bid and Ask do not have to be identical when a trade occurs. Someone may lift the offer and that results in a trade without ever having bid the asset b) The last trade may have occurred some time ago as pointed out and true value will most likely lie somewhere between ...

2

Try the FAST Antenna Codec for Windows. You need a little wrapper app that reads the UDP data from your dump and feeds the payload (i.e. sans IP/UDP headers) to the codec.

2

I am using a native C++ implementation based on QuickFAST, you can use the sample FileToMulticast to implement what you need. But if you are looking for a way do decode UMDF ( BM&FBOVESPA FIX/FAST) you must first take care with the UDP packages itself and re assembly the messages, before the decodification

2

To construct best bid/ask from ITCH you must build a book incrementally from the messages in the data. Every message, except for system oriented messages, and non-displayed Trades, represent an order or an action on an order. Process the data, build a book, and you will naturally be left with the best bid/ask at the top of each side.

Only top voted, non community-wiki answers of a minimum length are eligible