7
$\begingroup$

I would like to use tick data (especially order book data; but also trade) to backtest my strategies, can anyone offer a recommendation? Personally, I've written my own backtesting scripts couple of times, and also tried out backtrader. I'm aware of other Python backtesting tools out there, but typically they deal with OHLC data only.

I'm also aware of backtrader's Data Replay feature. Ideally, however, I would like to use actual order book data instead of simulated activity to analyse if we were able to execute the trade. In essence, my intended strategy is bordering slightly on cross-exchange arb/hedging execution—thus ability to execute in a timely fashion is quite critical.

From looking at other posts, I also understand that a couple other ways to deal with this include:

  1. Simulating order flow distribution or model transaction costs (both based on historical data).
  2. Using an OHLC backtester, but applied at a very minute interval (e.g. tick level). In this event however we are limited only to prices at the top of the order book (L1 data) and cannot evaluate our strategy properly?

I personally view these as next-best alternatives compared to backtesting with actual order book data. For those who have had experience in this aspect, do you mind sharing more on the steps you took to backtest your strategies? Or potentially suggestions that I may be blind to?

Thank you very much in advance!

$\endgroup$

4 Answers 4

8
$\begingroup$

Since order book granularity backtesting is challenging, as you've pointed out, I recommend first deciding your business requirements:

  1. Can you rely on a third party execution desk? I do not recommend modeling the transaction costs as a static number of bps, but the one exception is if all you're doing is dropping a large parent order and someone else is coming up with the execution trajectory on your behalf (this could be some other team within a company that is focused on execution research OR a bank's algo execution service).

  2. Does your strategy need to use passive orders? A strictly liquidity-taking strategy is easier to backtest.

  3. If your strategy uses passive orders, how important are your passive fills relative to the alpha that you're capturing?

Then, the most common 4 approaches are described below, in increasing order of difficulty.

Approach 1 (liquidity-taking only)

The best scenario is if you are only running a liquidity-taking strategy. Then, you could skirt the issue even with only top of book data: the practical solution is to backtest the strategy with a min size that is available at touch, put the strategy into production, then ramp up the size by hand, and calibrating for capacity based on real world PnL.

Another reason this tends to work well is that size has a self-fulfilling effect for liquidity-taking strategies - modulo liquidity replenishment effects, your PnL should get better with larger size because you're dislocating the spread further in your trade direction.

Approach 2 (conservative fills with MBP or consolidated top of book)

If you have to use passive orders, then the next best approach is to assume that the price that your order is on must trade through completely, i.e. a trade must post on the next level, to simulate a fill.

This takes into account the effect of hidden or max show orders etc. This is imperfect, since it is conservative on the fill probability. If the alpha that you're capturing is mostly from providing liquidity, then this usually doesn't work since it will underestimate the fills with adverse selection. On the other hand, if you have some longer horizon alphas and operate on a lower frequency, and passive orders are only being used to reduce slippage, this still works fairly well for backtesting and in practice.

Approach 3a (conservative MBP simulation)

To go one step further in sophistication, you'd need market-by-price (MBP) data. Then, you can just assume that all adds go behind your order while all cancels go to the backmost of a level.

This works well in practice because orders closer to the front have more optionality and tend not to be canceled.

Approach 3b (optimistic MBP simulation by modeling queue position)

Rather than conservatively assuming that all cancels go to the back, you can get more sophisticated by modeling your queue position from MBP data.

This is more sophisticated than 3a, but sophistication is not always a good thing - and I don't recommend this approach because it is hard to get right, and for the time you spend on it, you might as well invest it in Approach 4.

An anecdote in recent history can be found on CME, which was one of the most notable markets to shift from MBP to MBO. Some sophisticated trading firms had an edge from modeling queue position on MBP data, but that trade almost instantly vanished after MBO was introduced. This goes to show why Approach 4 is superior to Approach 3b. Otherwise, 3b is a fun academic project or the sort of thing you'd assign college interns at a prop trading firm.

Approach 4 (using MBO data)

Finally, you can just get your explicit queue position by backtesting with MBO data.

The most important thing to keep in mind here is that only a few vendors provide MBO data with the right timestamping techniques for this kind of simulation. (Disclaimer: The firm I work for, Databento is one of them.)

Most firms will market "high-frequency TAQ data" that is simply top of book from the SIPs, because it's much cheaper to distribute the SIPs. Other firms will market "L2" data that is just MBP data. And for the remaining firms that do provide MBO data, they often don't use appropriate timestamping techniques.

This then shifts your initial attention to the implementation of a LOB structure, which is the trivial part.

What most of your attention then goes to is (i) estimating matching engine round trip times and (ii) calibrating your estimate based on real world fills. The right timestamping techniques come into play especially because of (i).

$\endgroup$
1
$\begingroup$

I'm making a backtester as described in http://www.math.ualberta.ca/~cfrei/PIMS/Almgren5.pdf (p40) and it's very similar to what @rkr explained in 3a.

Please check it out if you're interested. https://github.com/nkaz001/hftbacktest

Now I'm finding a way for MBP simulation by modeling queue position(3b). These two posts might have a little more detail about it.

https://rigtorp.se/2013/06/08/estimating-order-queue-position.html

how do we estimate position of our order in order book?

$\endgroup$
5
  • $\begingroup$ If you ever need MBO or MBP data for traditional asset classes, feel free to try databento.com — we're in beta stage and if you mention that you were referred over from QSE, I can get you bumped up the waitlist for a credit. $\endgroup$
    – databento
    Nov 19, 2022 at 16:44
  • $\begingroup$ Thanks! MBO is still pretty expensive. Is it possible to buy it for a week or days? What's a plan for other markets such as JPX, and HKEX? These should be much cheaper so more accessible. $\endgroup$
    – kaz
    Nov 21, 2022 at 14:41
  • $\begingroup$ Yes we sell MBO data down to 10 minute intervals, but usually one should get a whole day at least. We're actually planning on releasing HKEX and JPX in 2023. $\endgroup$
    – databento
    Nov 22, 2022 at 9:05
  • $\begingroup$ That sounds good. I will check it out. $\endgroup$
    – kaz
    Nov 22, 2022 at 11:18
  • $\begingroup$ Cheers, I hope you'll find it useful $\endgroup$
    – databento
    Nov 22, 2022 at 23:41
-1
$\begingroup$

I would suggest Metatrader or cTrader for testing trades (cTrader far better for data availability already in the platform). For the order book part I have no experience on that, so I don't really want to speculate about it.

In order to test your strategies you will have to write them in mql4 / 5 regarding Metatrader or in cAlgo to be able to use them in cTrader

$\endgroup$
-3
$\begingroup$

This question is more concerned with a high frequency TAQ-like data source than a backtesting engine (which you can write your own when you have data).

In the book Trades, Quotes and Prices, the authors use the LOBSTERS limit order book academic dataset. They even offer a sample dataset with the book and 10 levels bids and asks are provided. You do have to pay for it if you want access. But the price is significantly cheaper than a commercial vendor like Exegy (it sounds to me that you are an academic user).

Otherwise, I am aware of Polygon.io. They offer trade and quote data too on a monthly subscription basis. But I have never used them before. From feedback, I think they are doing a very good job.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.