0
$\begingroup$

Let's say that there's some asset traded on an exchange and that, for this asset, I have access to a snapshot of the limit order book (price level and quantity for bids and offers) and subsequent updates to the order book of this asset allowing me to reconstruct an approximation to the order book at a point in time as well as trades as they occur. For simplicity, let us also assume that there aren't any hidden orders or other complicating factors.

Is there a common way to combine the streams of order book and trade data into something more homogeneous?

Since in one direction, the trades do hold important information about the order book such as allowing us to determine when a change in the quantity at price level of the order book is likely due to orders being matched or order cancellation and I can't think of an example, but I'm assuming the converse holds as well for information content.

Also, on a related note, what are some of examples of the target variable that a high frequency market-taker may try to model/forecast to trade upon? Are these just things such as the mid and micro price + a potential "spread" with any more informative constructions likely being proprietary?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Usually you need orderbook snapshot and update data to take into account latency, slippage, market impact etc. All these things are time sensitive and data intensive. The orderbook data itself is large, and you would have to retrieve that in milliseconds timestamps and do something with it. Basically, this process becomes very technical.

First step is to clean this data (get rid of out of order updates, take into account N levels and drop the rest, etc.).

Store the data in time series database. The way you store the data will impact how you retrieve it. Your data probably looks like: Snapshot -> Update -> Update -> ... -> Snapshot. One way would be, to construct snapshot from updates for every N millisecond and store only snapshot. You can do the same for trade data and retrieve based on timestamp. However, this approach is resource intensive and backtest would take a lot of time. A more optimal approach would be to use some OOP technics and store the data as classes. This will give you some freedom to store only updates, combine trade data and use various methods when retrieving the data. These approach will require some dev work but will save resources and makes is easier to use.

I did not understand your second question.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the in-depth response, I will piece through this regarding handling of the order book and trade data, regarding the second, I tried to clarify what I was asking, let me know if I'm still not conveying well enough. $\endgroup$
    – QMath
    Commented Aug 30, 2023 at 2:49
  • 1
    $\begingroup$ @QMath Arguably, orderbook data is the most essential data for HFT and market makers as a source of price discovery. The arrival of new orders, the imbalance, the depth, spreads etc. can be used as features for models and decision making. $\endgroup$
    – quantinho
    Commented Aug 30, 2023 at 4:37
  • 1
    $\begingroup$ I haven't done this but having thought about it before this is exactly how I would data enginner my solution. Also I would design algorithms around this structrue of data as well, rather than building an algorithm versus an individual snapshot, which is derived from a snapshot + updates, my algorithm would primarily respond to the data both in the last snapshot and the updates. Its a fine distinction but I think that would work better. Have you seen any good write ups on anything like this? $\endgroup$
    – Attack68 ♦
    Commented Aug 30, 2023 at 12:15
  • 2
    $\begingroup$ @Attack68 this could be a good reference: kb.timebase.info/community/overview/messages $\endgroup$
    – quantinho
    Commented Sep 4, 2023 at 3:15
  • $\begingroup$ Unsure if this warrants another question, but do either of you @Attack68 know of a common way to have the most up to date order book whenever a new message is received? Say we receive messages either in the form of orderbook updates or of trades that have occurred, how do we make sure that any any time, our order book reflects all available (to us) information without double counting? e.g. we receive a message that a trade occurred so we go to our order book and update the level that the trade occurred at. Then, we receive an order book update that among other updates, includes the update... $\endgroup$
    – QMath
    Commented Sep 7, 2023 at 6:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.