4
$\begingroup$

I'm wondering if some of you known how to resolve this requirement:

I have to define the architecture of an algorithmic trading system (but I'm not an architect, so I'm trying to do my best). I have defined an initial architecture, but just now I'm stuck at the data feed handler component.

I mean, the system will receive market data from different sources (bloomberg, FIX4.4, etc) and must normalize that data to produce an usable data feed which it's supposed to be consumed by algorithms to make some calculations an create some orders, something like this:

mk data providers => data component => normalize data => usable data => algorithms (consume normalized data)

So, I'm wondering if you know a good way to make this or maybe you know a good opensource market data feed handler that can receive data from different providers and produce one clean and normalized stream of market data.

I will appreciate very much your answer. Thanks in advance.

PD: I have been doing some research and I found this:

And for now I'm just reviewing those sites...

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

If I understand your question correctly, you're asking what's a good design for a normalized feed. This is a somewhat trivial question of (i) picking which data fields (e.g. price, volume) to filter out from each feed and (ii) how to keep that in a trading system with minimal computational overhead.

Regarding (i)

I highly recommend you approach this in a test-driven manner. In other words, figure out what data your application is going to use downstream and 'reverse out' what you need to normalize upstream.

A trivial example: If your strategy just needs prices, there's probably little use in normalizing the match numbers. If your compliance and post-trade analysis is keyed by transaction times rather than sequence numbers, then you probably don't need that either.

Two tips that I can give you are:

  • Don't overstretch yourself. It's easy to make the mistake of trying to normalize too many from day 1. You don't want to waste time trying to get spot FX, swaps, equity options, equities, exchange-traded futures etc. all onto the same normalized feed. It will probably take you a very long time before you're dealing with all that, and by the time you have to, you probably have someone else redoing this for you anyway.
  • Think a bit about how you will maintain depreciated fields and version changes you make to your normalized feed. No matter how exhaustive your initial design, I guarantee you will encounter changes over the course of implementation and real-time use.

Regarding (ii)

Without knowing your exact architecture, the best answer we can give you is a collection of obvious software development tips: keep this in memory, minimize allocation and GC overhead, and minimize the number of times you're shuttling the data from thread to thread.

Important note

If you're trading based on more than 1 data feed, this is not something you want to do alone. It's seriously not worth your time. There's a reason why vendor normalized data feeds are costly, they take a lot of time to maintain.

The part I don't get about your question is: Bloomberg is a normalized data feed provider whereas FIX is sometimes used by exchanges directly, why are you normalizing across the two? Then you're just introducing unnecessary overhead in the Bloomberg path. I would just use Bloomberg alone if that's the case.

Improve this answer
$\endgroup$
0
$\begingroup$

You should check CoralMD which does exactly what you need:

  • It provides a fast market data book implementation that can aggregate quotes from multiple exchanges together, giving you a global view of the market.

  • The market data book can also provide a per-exchange view.

  • It provides a simple framework to build market data feeds for each exchange.

  • And most important, it provides the infra-structure to distribute normalized market data internally for all parts of your trading system, from GUIs to low-latency strategies, as the diagram below illustrates:

enter image description here

Disclaimer: I am one of the developers of CoralMD

Improve this answer
$\endgroup$
3
  • $\begingroup$ CoralMD seems to be a great solution, but unfortunately I can't afford the product, so I would prefer build it by myself (but I don't know how to do it) or use an open source solution. Thank you very much :) $\endgroup$
    – Néstor M.
    Mar 6, 2015 at 17:51
  • $\begingroup$ I don't think "look at my software product" should be submitted to answer a question that begins "how do I ..." Because it's not an answer. On the other hand, that's not a bad comment and the product looks interesting. $\endgroup$
    – Nathan S.
    Mar 9, 2015 at 1:59
  • $\begingroup$ what's the minimum latency that CoralMD can handle? $\endgroup$
    – Ariel Silahian
    Feb 19, 2016 at 5:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.