How to make a trading universe of liquid futures contracts

Question

I am forming a universe of liquid futures/liquid FX forwards. I want a list of all liquid contracts, the key word being liquid. This is for an academic project, but you could imagine liquid being loosely defined as securities that could form the core trading portfolio of a mid-sized systematic trend-following CTA. This question is not meant to be a debate about the liquidity of individual contracts or markets- I'm asking for a systematic, repeatable procedure for determining a list of what I expect to be around 100-300 markets.

I am flexible in terms of how I form this list, provided the method is systematic. For example, one approach might be to rank all generic contracts in Bloomberg by (say) 100-day trailing ADV as of (say) EOY 2019. Unfortunately, I have no idea how to do this, and help desk didn't seem to have a solution. Another issue here is narrowing down the list of securities that I am ranking such that I don't trip the Bloomberg API limits (for those less familiar with Bloomberg, this means I'd be hesitant to query more than 1000 or so securities).

With equities, I would get a list of all stocks from the NYSE/NASDAQ/AMEX. But on the CME website, there seem to be a lot of contracts without listed volume, so this approach seems inappropriate.

Another possible way might be to include the constituents of 1 or more futures indices. For instance, with equities I would use the Russell 3000. However, I haven't found an analogous index for commodity futures. BCOM constituents form a very small and limited list (for instance, they do not include Financials). GSCI is another small list. But a broader list seems hard to find.

A word on data sources: I have access to Bloomberg and limited access to WRDS. My preference is for Bloomberg since the generic contracts are particularly convenient, though obviously I can make a crosswalk if necessary.

Answer 1

Systematically finding most liquid futures instruments

---

Can we put together a better list than the academic articles?

Yes! The lists in existing publications [1, 2] are great, but fall slightly short of your goal:

I'm asking for a systematic, repeatable procedure for determining a list of what I expect to be around 100-300 markets instruments. [3]

What's liquid in 2012 and 2016 might not be in 2020. For example, the Micro E-mini S&Ps didn't launch until 2019, but are currently more active than many of the instruments in those academic publications. Moreover, you can't control the number of instruments to qualify. For these reasons, generating them systematically is a lot better.

Using secdef files

One strategy is to use open interest as a proxy for liquidity instead of volume. Then, you can use secdef files which CME updates regularly on their public FTP server here.

The secdef files are written in plaintext and can be parsed like any FIX data. You can see a dictionary of all the fields here. In your case, you're probably interested in 207=SecurityExchange, 1151=SecurityGroup, 55=Symbol, 167=SecurityType, 462=UnderlyingProduct, 5792=OpenInterestQty. You'll need decide if you want to aggregate the open interest across all contract months in ranking your instruments. To make it simple, I assume that you do in my example code, but my solution can be extended easily.

If you're not familiar with the Globex product codes, you can use the product slate.

Example code

I made example code available on GitHub to demonstrate how this is done. Here's the top 10:

XCME,GE,Interest Rate,10743955
XCBT,ZF,Interest Rate,3618711
XCBT,ZN,Interest Rate,3391115
XCME,ES,Equity,3265839
XNYM,CL,Energy,3003110
XCBT,ZT,Interest Rate,2447405
XNYM,NG,Energy,2217482
XCBT,ZS,Commodity/Agriculture,1729122
XCBT,ZQ,Interest Rate,1719442
XCBT,ZC,Commodity/Agriculture,1395498

This approach lets you rank instruments and easily qualifies a larger set of instruments (70+) including the Micro E-mini S&Ps. It also lets you test the stability of your selection. You'll just need to store the secdef files over time by yourself. But for your convenience, you can get a free batch of historical secdef files from Dec 2019, hosted by Databento* here, together with a complete list of 163 instrument groups generated from the 05/11/2020 secdef file.

For full disclosure: I work at said firm.

---

Other issues

Also, to fill the gaps on 2 issues you're experiencing:

Another issue here is narrowing down the list of securities that I am ranking such that I don't trip the Bloomberg API limits (for those less familiar with Bloomberg, this means I'd be hesitant to query more than 1000 or so securities).

The generic terminal isn't designed for systematic querying and ranking of instruments across the entire market. To do this with Bloomberg, you'll likely need a B-PIPE subscription instead.

One approach might be to rank all generic contracts in Bloomberg by (say) 100-day trailing ADV as of (say) EOY 2019.

This isn't easy because futures contracts expire. You will need some systematic way to map from the individual contract months (e.g. ESM0, ESU0) to their root symbol (ES), otherwise you won't be able to compute ADV over a long horizon like 100 days. You might be able to get away by discretizing into month buckets, but you'll still need to account for the fact that rollover dates, lead months and active months, differ from contract to contract.

Answer 2

You could consider using the list of "liquid futures contracts" used in some previously published paper(s) on this subject, there are many. Alternatively, if you think previous studies missed some important contracts you could try to establish your own list independently.

I thought for example of the using the following from a well know paper:

Moskowitz, Ooi and Pedersen (2012) Appendix A

And you have also suggested two others:

Koijen Moskowitz Pederson Vrugt (2018) Appendix D

Han Hu Yang (2016) Table 1