this is actually a lot more difficult to write than a
backtest for market orders
This is to be expected. In my experience, the engineering difficulty and time goes up 1-2 orders of magnitude, because, among many things:
- On the market simulation side, you need
- full order book data, instead of BBO only;
- much more precise timestamping on your data;
- much more bookkeeping of strategy orders;
- much finer measurement of various matching engine, gateway, and client side latencies;
- to account for at least an order of magnitude more obscure matching scenarios and edge cases like auctions, halts, RFQs, non-pure price-time priority, implied book;
- to deal with unobservable behavior like adverse selection, hidden liquidity;
- simulate self-exciting effects that would be present in production when your orders actually get published in the feed;
- tooling for post-trade calibration of your simulation.
- On the client side, you now need to manage a much larger state space.
Firms that excel at simulation-based trading dedicate tens of man years to fine-tune it, and have specialists for writing simulation targeting specific venues. Even some of the top 30 market makers don't have good passive simulation.
I would like to know if anyone has done limit orders backtesting? Any advices? Or this is actually a very bad idea ?
Also see my response here. I generally recommend against it unless the principal purpose of your strategy is to provide liquidity and tighten the spread. Otherwise the next best reason is that you're using limit orders purely to reduce slippage, but your strategy is already passable (and in production for some time) with market orders.
The worst reason to be considering it is if your strategy has still not been deployed to production and you're incorporating passive, non-marketable orders to make it pass whatever concrete or subjective threshold you have for deploying the strategy.
because the backtest needs to take into
account of when the limit order got hit
In addition, if I set a limit to the position that i am allowed to take,
this makes the backtest very difficult to write.
You're not wrong regarding both statements here. It's entirely a matter of implementation difficulty. That said, both of these are among the easier problems with passive simulation actually:
- You can make a first order estimate (this will be imprecise) when the "limit order got hit" by maintaining a separate order book structure on the simulator side, and ack'ing a fill to the strategy when a trade takes place on your stored book data with lesser priority than your order.
- Most strategies should have position limits, even liquidity-taking strategies. Even if not for model or risk management purposes, you'd still need a position limit due to margin requirements. If your platform is well-designed, the same pre-trade risk layer will handle both simulation and production, and part of this - the strategy still needs a position exceeded failure state - will be modularized away from your strategy logic.
