# Performance of Open Source Time Series Database for Financial Market Data

We would like to store financial tick data in a database (potentially billions of rows) and then create aggregated (open-high-low-close) bar data from it (e.g. 1min or 5min bars).

It was mentioned to us that a NoSQL or time series database might be a good choice for this. Can anybody give any advice on which open source product might fit this requirement best.

Note: query performance is very important for us.

In our research we came across the following products (maybe there are more):

We did run a test with InfluxDB with around 10 million ticks. Unfortunately the creation of 1min bars was 3-5 slower than with a relation database (i.e. MySQL).

We are aware that KDB now offers a free 32-bit version, but unfortunately 32-bit will not be enough for our use case.

EDIT (Sept 2015): We also did a test with OpenTSDB which seems to be quite fast. The import of 10 mio. prices took about one minute and the aggregation into 1 Min Bars took about 5 seconds.

EDIT (Jan 2017): More than one year after the initial test we gave InfluxDB another try and it turns out that they have made huge progress in the meantime. Write performance is now up to 2 mio. data points per second (with version 1.2)! We have now decided to integrate InfluxDB into our own product AlgoTrader

• InfluxDB doesn't provides out-of-the-box Technical Analysis functions so how do you perform indicators calculation on your timeseries data ? thanks – Florent May 6 '18 at 6:46
• That is correct, you have to feed/stream data into your own system and do the analysis there. The is how we do it on our side – Andy Flury May 7 '18 at 11:29
• I asked the question to their community and apparently Kapacitor is a data processing engine that gives the possibility to calculate custom indicators in the database. I have no idea if it is faster than doing it in a Node JS server served by a Mongodb for example - which is my plan. At least it's going to be much more easy. – Florent May 7 '18 at 12:37

You could try Arctic. Other open source column-oriented databases that you may not have considered include LucidDB and C-Store.

• Thanks! Could you add a link to Arctic? The link above actually points back to this question. I'm aware of LucidDB and C-Store, but it looks like both are dead. May main question was which of the mentioned databases would actually produce the best performance. – Andy Flury Sep 5 '15 at 10:02
• @AndyFlury Fixed. – madilyn Sep 5 '15 at 20:54
• Interesting! Looks like this is related to this video from ManAHL: youtube.com/watch?v=FVyIxdxsyok – Andy Flury Sep 6 '15 at 13:23
• Yes, Arctic is AHL's tick and TimeSeries data store. We've presented a few times on its implementation: github.com/manahl/arctic/wiki Performance (from Python) is good - millions of rows/s. Happy to answer any questions! – James Blackburn Sep 15 '15 at 12:41

(I work for Axibase)

Axibase Time Series Database is not open-source but it's free on single node.

Time precision is microseconds.

It supports OLCHV+VWAP aggregators in SQL and REST API with various functions for filtering by trading calendar, auction stages, indices etc.

SELECT datetime, symbol, close(), vwap()
WHERE in_index('<index-name>')
AND in_session(DAY, CLOSING)
AND datetime BETWEEN '2021-01-01' AND '2021-01-15' EXCL
GROUP BY exchange, class, symbol, PERIOD(1 DAY)


(I work Axibase)

• Thanks Sergei, interesting product! You are the owner of Axibase right? You should always disclose this information as they are very picky about advertising products in response to questions here on stackexchange. Looks like you are using similar technologies to OpenTSDB which produces similar performance figures too (see my comment on the main question). – Andy Flury Sep 21 '15 at 8:30
• @AndyFlury Thanks for pointing it out. Disclosure added. – Sergei Rodionov Sep 21 '15 at 8:36

Take a look at Cassandra. Free and Open Source DB, noSQL. It almost perfectly fits your case.

• Thanks @AlexZeDim. Cassandra is a good NoSQL database. However it is not specifically built for time series data, so you would have to add a lot of that functionality on top yourself. As far as I know OpenTSDB can be used on top of Cassandra – Andy Flury Jan 29 '17 at 12:12

** disclosure: I work for quasardb **

Hi - you may want to run the community edition of quasardb. If your dataset if small enough (32 GB storage) - this may very well work !