21

Here are some pointers. First of all: What you list as a Reuters RIC, RSF.ANY.AAPL.OQ, is not really a RIC, only the AAPL.OQ is. The initial part is some stuff which is essentially site specific and tells me that you are working on a site that has a legacy RTIC infrastructure (some Reuters/TIBCO technology which is quite old these days and for all ...


19

The main issue measuring intraday volatility is called "signature plot": when you zoom in, the volatility measure (i.e. empirical quadratic variations) explode. Similarly you have the "Epps effect" for correlations: when you zoom in, the correlations collapse (it is at least a mechanical effect). For the volatility a lot of models can correct this: - first ...


14

The expression you have is fine. But more generally, for the intraday volatility, I don't think there "the correct definition". More like, whatever works in the given context. I found the following notes by Almgren pretty useful: http://cims.nyu.edu/~almgren/timeseries/notes7.pdf


14

There are many specialised products for HF tick data. In addition to KDB which you mentioned, there is OneTick, Vertica, Infobright, and some open-source ones like MonetDB etc. (see http://en.wikipedia.org/wiki/Column-oriented_DBMS). My experience is that Column Oriented Databases are overrated when it comes to tick data, because very often you request the ...


13

Your question is very vague (e.g. what are you trying to measure, and what "tick data" do you have), but I'll give you some pointers: In general, when people consider how prices evolve, they will tend to think about things like volatility and correlation dynamics. So I would start by defining exactly what you want to measure. The irregularity of time ...


11

We use Node for reporting but not as part of our main signal generating trading system. To be honest the answer will almost certainly be yes for every common programming technology as it just takes one person to use it somewhere to make the answer yes. Just look at OCaml, before Jane street, most techno logiest on the street had never heard of it and now ...


8

OpenTSDB is good for large-scale time series storage. metrilyx/opentsdb-pandas and wiktorski/opentsdb_pandas seems to provide the interface with pandas. OpenTSDB and HBase rough performance test | MoreDevs provides a benchmark, may not exactly match your requirements but you can try.


6

Using MySQL for financial data is not unreasonable. But for tick data are you ever going to do anything except a query on a date range? For analyzing tick data in R I generally keep it in a disk file, one tick file per day, and load the files in as I need them. Using .RData files instead of csv files is quicker. I've also used custom C++ classes before, to ...


6

Statistical volatility is the standard deviation of a window of log returns. For example, 30-day statistical volatility is the standard deviation of 30, one-day log returns. The log return comes from the assumption that log stock returns are normally distributed. Statistical volatility differs from implied volatility which is the volatility input to some ...


6

On a theoretical level and for low frequency data (e.g. daily), your formula seems right. However, since you are talking about one minute bars, things may get a little messy. There is a vast literature on this, and empirically, things are complicated due to market micro-structure noise. Namely, you need to do consider jumps, errors, periods of low volume, ...


6

Cloud9Trader uses Node.js on the back end and JavaScript across its technology stack, including for writing the trading algorithms themselves. https://www.cloud9trader.com


6

Dukascopy offers historical tick data. Through their historical data website you can download what you want, but registration is required, and lots of manual clicking. However if you are comfortable with scripting, you can directly download the tick data yourself. The URL pattern is http://www.dukascopy.com/datafeed/{currency}/{year}/{month}/{day}/{hour}...


6

As someone who has contributed to literature, I am purposefully vague with the use of mid price. Not that I don't define it but that it is difficult to state which definition is the best in which context. Here are an example of a few definitions of mid price: Last Trade: The physical price at which the most recent trade physically took place. This is ...


5

Your code for volatility seems correct, if you want minute volatility, but is that really what you want? See this recent question on annualizing volatility from intraday data. Also, using first and last tick is what is generally done, but over very short time intervals such as a minute, you will have microstructure issues. Another question here deals with ...


5

xignite seems like a cheaper option for historical data at around $6,000 per year. I am not a client (yet) so I can't give my opinion but it looks good to me. They have a long list of services, a free trial period and interesting pricing model based on the number of queries you send them. If you do use them, let us all know.


5

I think you're addressing the wrong people. What I'd do is to talk to the data vendors' marketing departments, much like I'm sure http://www.wikiposit.org/ has done. Tell them you would like to offer (an excerpt of) their data, in a unified/post-processed form, you would be willing to include links back to their sales website, or praise the vendors somehow,...


5

Collect the data yourself and self-timestamp it or buy it from a vendor that timestamps on receipt. If you do buy data from the venues you should consider very strongly the fact that venue timestamps are never in agreement. Therefore, an event that INET reports at time A does not coincide with an event that BZX reports at time A. This makes buying this sort ...


5

Yes, there is a software application that you can purchase for $39.99 which stores all your tick data in a highly compressed format while still allowing maximum throughput and lowest latency data queries that I have ever seen. The package provides APIs to all languages under the sun but because they have a special sale going on it comes with the complete ...


5

Obviously merging two streams is harmless and it should be done. But it's hard to advise you regarding the "interpolation" methods you can use to generate the ticks without knowing why you need this. The reason is that any method will introduce a certain bias to the data. Therefore, it very much depends on what are you going to do with your altered data on ...


5

If you're missing ticks, then no technique will get those ticks back. If you have two sources, then designate one source as the primary feed and then fill-in gaps from the secondary feed. Of course, you'll have to mind the timestamps when determining whether the secondary feed can be used properly.


5

I think the best choice for technical analysis with node is node-talib, a wrapper around TA-Lib. We're using it for some projects and it works ok so far. Here's a list of the indicators you get out of the box: AD Chaikin A/D Line ADOSC Chaikin A/D Oscillator ADX Average Directional Movement Index ADXR ...


4

Bloomberg equity codes are usually quite easy to derive if you know the ticker (though you may have to replace dashes/spaces/slashes for preferred and multi-class shares). This is just for equities though and this is definitely not the case for futures. RICs are a different story. In the US, you need to know on what exchange the ticker is listed to get the ...


4

As explained in the comments best bid and best offer (best ask) are the best prices at which you can respectively sell and buy at least one unit of the asset your are considering. When backtesting a strategy, most people usually either use best bid and best offer or even worse last price. The problem is that these prices are only available for a limited ...


4

I use Yhang Zhang measure for intraday volatility for timeseries with a rolling 5 or 10 day window. I wrote a C++ and vba implementation which I'm happy to share if you wish. Takes olhc data and gives an 'estimate' of the volatility. For intraday trading (gamma hedging), I found it is a fairly good estimator of the days range. But I would caution on whether ...


4

I'm trying to formulate an answer for latter users of the site and, hopefully to lure some more experienced high frequency traders to further react to my post. First of all, I'd notice that the paper you mentioned was published in 1991. I believe high-frequency trading has evolved a lot since then, and that nowadays HF strategies are more evolved. From ...


4

It turns out that the Bloomberg Terminal QR function, when adjusting the timezone from Exchange/UTC to your timezone, will convert the time but not the day. Trades displayed via IntradayTickRequest API are correct in UTC time, trades displayed via Bloomberg Terminal QR may be incorrect due to failure to adjust the stated date for timezone adjustments.


3

It isn't clear from this post or your last post what data you want and what you want it for. The only thing you are going to get permission to distribute is the higher level data feeds like EOD data, or last sale (probably can't offer history). Exchanges typically have this information available on their websites, for instance NASDAQ Basic pricing. If you ...


3

I use interactive brokers as well. What I do is I collect the data throughout the day with their API (reqMktDepth functionality) and I'm building my personal data warehouse. It may seem silly but I've been doing it for two years now and I have a good dataset to play with. Just get started and you'll see that it makes sense, the nice thing about it is that ...


3

Each vendor has their own symbology universe and each exchange, market, or country may have their own standard identifiers. To date the majority of identifiers have also been dynamic, i.e. when a company renames, merges, relocates, the identifier may change. Some symbology systems operate on different levels of granularity whereby prices can be filtered to ...


3

If you just want to run some simplistic technical analysis on quotes, then select the last quote for each unique timestamp. That will ensure that you don't have duplicate timestamps. If you must have it evenly spaced (i.e. no gaps from one second to another), then you can reuse the previous quote to fill-in the missing value.


Only top voted, non community-wiki answers of a minimum length are eligible