Limit and Market Order for training a ML model

Question

Goal : Using deep learning to build a ML model which would predict the right places where a stock price will increase, decrease or stay stable.

For the current question, assume the labels are well defined.

As features I used the limit order book, but I did not get a lot of success. So I realised that I didn't have all the features to accomplish my goal. In reading the following paper https://arxiv.org/pdf/1710.03870.pdf on page 8, I didn't consider the market and cancellation orders. A setback I got is I did not find a vendor to provide the market orders separately from the limit orders. So in the moment I have the full market depth from algoseek. You can find a sample here.

Does the full market depth contain the market orders? If so, can I build just a book containing the limit and market orders I can use for the training phase of the ML model? Do we really need to separate the limit and market orders to feed the ML model.

Please let me know if you question is unclear.

Answer 1

You are right that there are limit and market orders and nothing else. But I'll clarify it. Market orders do not stay in the order book because they instantaneously matched against limit orders and generate trades. Limit orders depending on their limit price can also be instantly matched fully or partially. So, do you refer to aggressive orders in general (limit or market) or orders that are Market by type?

In any case, each exchange has its own way / data format to inform about it. Plus, data vendors (e.g. algoseek) may also post-process the data and convert it to another format. In this case it's necessary because algoseek needs to provide the data of the same symbols from different exchanges in a uniform format. As you can see below they don't do it very effectively, but it's up to them.

The documentation of the sample data seems to be outdated, but it's easy to guess the format from the content. So, in order to get Market orders or aggressive limit orders from the data or you need to analyse the trades. Here is an example. To make it more compact I removed the columns of Exchange, MMID, and Symbol.

If you wish to see actual Market orders in the data when they arrive, check GDAX data, see the description of the full channel in the API. Here is a short sample of the raw data. Each line starts with a timestamp (set by recording device).

Also, I highly recommend reading this crash course on Market Mechanics to clarify all the above and much more.

To summarize, any trade that you see in the market data is caused either by a Market order or by an aggressive (crossing the spread) Limit order. Basically, you can model a Market order as an aggressive Limit order with limit price deep in the opposite side.

In some cases like with GDAX data you can see explicitly the type of the order that caused a trade, in some cases you can't see it. And in some cases you can back-engineer they type of the order based on the sequence of data updates. It allows to distinguish partially executed limit orders from market orders and fully executed limit orders. See case #4 from top on this diagram from the Market Mechanics article: