55 votes

What is an efficient data structure to model order book?

The specifics depend on if you're implementing for equities (order-based) or futures (level-based). I recommend https://web.archive.org/web/20110219163448/http://howtohft.wordpress.com/2011/02/15/how-...
Charles Cooper's user avatar
25 votes
Accepted

Red Black Trees for Limit Order Book

Why do people suggest using red black trees/balanced binary trees for the levels in a limit order book? Because people are unoriginal and keep referencing the same blog post. Why are they ...
databento's user avatar
  • 2,468
12 votes

What is an efficient data structure to model order book?

I recently stumbled upon this question after needing to do this to do some real-time microstructure analysis and have taken a look at the various possible implementations. Here are some of the pros/...
alfalfasprout's user avatar
11 votes

How do I convert order book data into OHCL( Open,High,Low,close) format?

As Alex C. notes, OHLC bars are meant to be calculated using transaction ticks. However, you could try to make bars from bid/ask individually (or perhaps even the mean of the two as an approximation), ...
lagrange103's user avatar
10 votes
Accepted

Efficient way to store orderbook in Python

The best solution is to change how you're doing it completely: Store the data before it even gets to your program and process it later. e.g. Use your program merely as a daemon to subscribe to the ...
databento's user avatar
  • 2,468
9 votes
Accepted

Modelling queue position

No, you have to build your model empirically with data. Suppose $p(x)$ denotes the probability of cancel in front of you when your order is positioned $0 \leq x \leq 1$ through the queue, there are a ...
databento's user avatar
  • 2,468
9 votes
Accepted

Backtesting using microstructure (orderbook) data

Since order book granularity backtesting is challenging, as you've pointed out, I recommend first deciding your business requirements: Can you rely on a third party execution desk? I do not recommend ...
databento's user avatar
  • 2,468
8 votes
Accepted

Order Book Dynamics

The Queue Reactive Model (by Huang, L and Rosenbaum) is an improvement of what Cont and de Larrard (CL) did. This model is capturing the inflows and outflows in each queue given the current state of ...
lehalle's user avatar
  • 12k
8 votes
Accepted

How does one calibrate lambda in a Avellaneda-Stoikov market making problem?

(First of all, sorry to have taken so much time to see this question...) For the paper you refer to (Guéant-L-Tapia), there is a report (in French) by Sophie Laruelle about how to do it in practice: ...
lehalle's user avatar
  • 12k
8 votes
Accepted

How can one effectively approximate the fill portion of a limit order in a FIFO order book given it's recent state?

In my opinion, instead of developing an analytical model, it's better to evaluate this probability directly from the data. Place your simulated orders at different price levels, and check whether and ...
Serg's user avatar
  • 1,012
7 votes
Accepted

What are the proper ways to do order book downsampling?

Two chief reasons for subsampling or using a different event space are (i) computational or spatial tractability and (ii) denoising/signal extraction. Sergei's response seems to focus on the first ...
databento's user avatar
  • 2,468
6 votes
Accepted

Limit order book - does it get wiped over night?

Orders stay in order book for as long as you specify, e.g. "good till cancelled" (GTC) will be sitting there until it's executed or you cancel it. There are many types, just google them - IOC, FOK, ...
rbm's user avatar
  • 735
6 votes
Accepted

What are flickering orders?

A "flickering" order is one which is repeatedly submitted and cancelled (whether it's at the top of book or not). The answer from @chollida mentions that "the goal typically is to either slow down ...
Chris Taylor's user avatar
  • 5,901
6 votes

What is an efficient data structure to model order book?

Curious how nobody in 9 years has provided an optimal answer yet? You need a B-Tree that has been extended with a double-linked list on each node. The B-Tree to find (or not find if they're missing) ...
Anon Coward's user avatar
6 votes
Accepted

What does a electronic dealer track in a RFQ market?

Customers dont place limit orders or do they? No, they don't. In an electronic RFQ market, the requesting participant (presumably the "customer" you are referring to) is generally not ...
databento's user avatar
  • 2,468
5 votes

What is an efficient data structure to model order book?

Why not just implement it with two price-amount maps, as shown here?
Dmitri Nesteruk's user avatar
5 votes

Identify Iceberg Orders

Liquidity seeking is difficult to test empirically since price only data tells us nothing about the how transactions related to the level 1 (NBBO) order book, let alone the depth of the order book. ...
David Addison's user avatar
5 votes
Accepted

Model reference price of Limit order book

This reference price is also sometimes called intrinsic price. One of the simplest ways to improve it in regards to the mid-price (assuming you have the depth data) is the following: define a ...
Serg's user avatar
  • 1,012
5 votes

Limit order book cancellations

This is somewhat of an opinion based response with some factual anecdotes at the end. Spoofing is a very difficult concept to define, identify and prove. In voice brokered markets where transactions ...
Attack68's user avatar
  • 9,974
5 votes

Red Black Trees for Limit Order Book

There is a difference about understanding LOB dynamics and using an algorithmic solution to capture these dynamics. How LOB evolves. We understood now long ago (see Jeremy Large's papers) that a ...
lehalle's user avatar
  • 12k
5 votes

Limit Order Book disaster recovery technique

For data on a given single host, a common architecture is to write the data onto a queue (could be multi-consumer or single consumer), and have an asynchronous writer that reads it off and persists it ...
databento's user avatar
  • 2,468
5 votes

Limit Order Book modeling

The Queue Reactive (QR) model (by Huang, L and Rosenbaum) is probably the most reliable to simulate order book dynamics. See this answer Order Book Dynamics for specific details. Let just say that the ...
lehalle's user avatar
  • 12k
4 votes

Limit order book - does it get wiped over night?

If you are processing ITCH messages, for instance, each day is separate, and starts with an empty book. If there are such things as GTC orders anymore, the ITCH feed would resend their Add messages ...
experquisite's user avatar
  • 1,182
4 votes

Implementing data-structures in a Limit order book

I am not familiar with F#. I have implemented this many times in C++. I would go with fixed length arrays. Fr me performance is paramount. In C++, one is better off handling holes than allocating ...
Humble Debugger's user avatar
4 votes

Estimate probability of limit order execution over a large time frame

One way to do this is a simple Monte-Carlo simulation. There are formulae you can use to get the likelihood of a stock being below a price if you know the stock's volatility and time frame - see for ...
Steinwolfe's user avatar
4 votes

What is an efficient data structure to model order book?

You can also utilize two red black trees. Bids would be stored in descending order in one tree. Asks would be stored in ascending order in the other. The best bid and ask would always be the first ...
Darwayne's user avatar
  • 141
4 votes
Accepted

Limit and Market Order for training a ML model

You are right that there are limit and market orders and nothing else. But I'll clarify it. Market orders do not stay in the order book because they instantaneously matched against limit orders and ...
Serg's user avatar
  • 1,012
4 votes

Modelling HFT data

By "modelling the LOB and the evolution of the LOB" you are modelling trades by definition as they are what determine how the LOB is going to behave. The evolution of the limit order book is ...
Theodore's user avatar
  • 1,172
4 votes
Accepted

Standard ways of simulating order books

Nowadays (4 years after your question...), the best way to simulate orderbook dynamics is probably to implement the Queue Reactive model, from Huang, Weibing, C-A L, and Mathieu Rosenbaum. "...
lehalle's user avatar
  • 12k

Only top scored, non community-wiki answers of a minimum length are eligible