Hot answers tagged tick-data
11
There are many specialised products for HF tick data. In addition to KDB which you mentioned, there is OneTick, Vertica, Infobright, and some open-source ones like MonetDB etc. (see http://en.wikipedia.org/wiki/Column-oriented_DBMS).
My experience is that Column Oriented Databases are overrated when it comes to tick data, because very often you request the ...
6
The expression you have is fine. But more generally, for the intraday volatility, I don't think there "the correct definition". More like, whatever works in the given context. I found the following notes by Almgren pretty useful:
http://cims.nyu.edu/~almgren/timeseries/notes7.pdf
6
The main issue measuring intraday volatility is called "signature plot": when you zoom in, the volatility measure (i.e. empirical quadratic variations) explode.
Similarly you have the "Epps effect" for correlations: when you zoom in, the correlations collapse (it is at least a mechanical effect).
For the volatility a lot of models can correct this:
- first ...
5
Yes, there is a software application that you can purchase for $39.99 which stores all your tick data in a highly compressed format while still allowing maximum throughput and lowest latency data queries that I have ever seen. The package provides APIs to all languages under the sun but because they have a special sale going on it comes with the complete ...
5
If you're missing ticks, then no technique will get those ticks back.
If you have two sources, then designate one source as the primary feed and then fill-in gaps from the secondary feed. Of course, you'll have to mind the timestamps when determining whether the secondary feed can be used properly.
5
Your question is very vague (e.g. what are you trying to measure, and what "tick data" do you have), but I'll give you some pointers:
In general, when people consider how prices evolve, they will tend to think about things like volatility and correlation dynamics. So I would start by defining exactly what you want to measure. The irregularity of time ...
4
xignite seems like a cheaper option for historical data at around $6,000 per year.
I am not a client (yet) so I can't give my opinion but it looks good to me.
They have a long list of services, a free trial period and interesting pricing model based on the number of queries you send them.
If you do use them, let us all know.
4
I think you're addressing the wrong people. What I'd do is to talk to the data vendors' marketing departments, much like I'm sure http://www.wikiposit.org/ has done. Tell them you would like to offer (an excerpt of) their data, in a unified/post-processed form, you would be willing to include links back to their sales website, or praise the vendors ...
4
Statistical volatility is the standard deviation of a window of log returns. For example, 30-day statistical volatility is the standard deviation of 30, one-day log returns. The log return comes from the assumption that log stock returns are normally distributed.
Statistical volatility differs from implied volatility which is the volatility input to some ...
4
As explained in the comments best bid and best offer (best ask) are the best prices at which you can respectively sell and buy at least one unit of the asset your are considering.
When backtesting a strategy, most people usually either use best bid and best offer or even worse last price.
The problem is that these prices are only available for a limited ...
4
Using MySQL for financial data is not unreasonable. But for tick data are you ever going to do anything except a query on a date range? For analyzing tick data in R I generally keep it in a disk file, one tick file per day, and load the files in as I need them. Using .RData files instead of csv files is quicker.
I've also used custom C++ classes before, to ...
3
Here are some pointers.
First of all: What you list as a Reuters RIC, RSF.ANY.AAPL.OQ, is not really a RIC, only the AAPL.OQ is. The initial part is some stuff which is essentially site specific and tells me that you are working on a site that has a legacy RTIC infrastructure (some Reuters/TIBCO technology which is quite old these days and for all ...
3
Obviously merging two streams is harmless and it should be done. But it's hard to advise you regarding the "interpolation" methods you can use to generate the ticks without knowing why you need this. The reason is that any method will introduce a certain bias to the data. Therefore, it very much depends on what are you going to do with your altered data on ...
3
Collect the data yourself and self-timestamp it or buy it from a vendor that timestamps on receipt.
If you do buy data from the venues you should consider very strongly the fact that venue timestamps are never in agreement. Therefore, an event that INET reports at time A does not coincide with an event that BZX reports at time A. This makes buying this sort ...
3
I use interactive brokers as well. What I do is I collect the data throughout the day with their API (reqMktDepth functionality) and I'm building my personal data warehouse. It may seem silly but I've been doing it for two years now and I have a good dataset to play with. Just get started and you'll see that it makes sense, the nice thing about it is that ...
3
I'm trying to formulate an answer for latter users of the site and, hopefully to lure some more experienced high frequency traders to further react to my post.
First of all, I'd notice that the paper you mentioned was published in 1991. I believe high-frequency trading has evolved a lot since then, and that nowadays HF strategies are more evolved.
From ...
3
On a theoretical level and for low frequency data (e.g. daily), your formula seems right. However, since you are talking about one minute bars, things may get a little messy. There is a vast literature on this, and empirically, things are complicated due to market micro-structure noise. Namely, you need to do consider jumps, errors, periods of low volume, ...
2
Your code for volatility seems correct, if you want minute volatility, but is that really what you want? See this recent question on annualizing volatility from intraday data. Also, using first and last tick is what is generally done, but over very short time intervals such as a minute, you will have microstructure issues. Another question here deals with ...
2
It isn't clear from this post or your last post what data you want and what you want it for. The only thing you are going to get permission to distribute is the higher level data feeds like EOD data, or last sale (probably can't offer history).
Exchanges typically have this information available on their websites, for instance NASDAQ Basic pricing. If you ...
2
This sounds like a case where you will need to apply some good old-fashioned judgment to determine what the standard deviation "should be" before you have enough data to measure it. Surely this process repeats with some frequency, and perhaps given some attributes and more details you could make an educated guess as to the standard deviation (or quantiles, ...
2
If you just want to run some simplistic technical analysis on quotes, then select the last quote for each unique timestamp. That will ensure that you don't have duplicate timestamps. If you must have it evenly spaced (i.e. no gaps from one second to another), then you can reuse the previous quote to fill-in the missing value.
2
I will be adding here some resources that I found during the research, hopefully it will turn useful for others:
http://www.linkedin.com/groups/master-symbol-database-most-cost-98173.S.51776671
2
Bloomberg equity codes are usually quite easy to derive if you know the ticker (though you may have to replace dashes/spaces/slashes for preferred and multi-class shares). This is just for equities though and this is definitely not the case for futures.
RICs are a different story. In the US, you need to know on what exchange the ticker is listed to get the ...
1
To help you understand why you need to follow recipes (like chrisaycock's) just have a look at your tick data. You will find ticks clustered at some points in time while they seem scarce at others.
If you proceed with your recipe 2, you will lose those clusters of activity and stretch them out. In periods of low activity you will condense the market.
...
1
Bloomberg provides unique identifiers, I assume Reuters does the same thing. It might be a good idea to use those, or at least track them. Presumably you have pricing data on each of these? You could always check a few close prices against each other to check if you have matched the strings correctly.
1
dxFeed does offer level 2 historical data on demand. You can try it out for free. For example, to see how all market makers pulled out their quotes on ACN (Accenture) during May 6, 2010 Flash Crash go to the following URL: http://demo.dxfeed.com:7070/onDemand/data?categories=E-M&symbols=ACN&start=20100506-1447&end=20100506-1449 and use demo/demo ...
1
As you said, this sort of financial data can be well aggregated using cubes; icCube is for example a fast in-memory OLAP server you can access via XMLA clients or JAVA or Javascript native API (on top of the MDX language and HTTP protocol).
[edit after comment] open/close/high/low are types of aggregation supported by icCube; so creating this kind of 'bars' ...
1
I have had success using MySQL to store both OHLCV, Options data and metadata such as earnings dates in MySQL and accessing both for reads and writes from R.
For me this works very nicely and is performant for daily data, if you are doing HFT you may want to consider a specialized tickdb, but at daily scales (252 returns per year per ticker - MySQL is ...
Only top voted, non community-wiki answers of a minimum length are eligible
