Is there a case to trade liquid OTC options (FX, single-name equity, swaptions, etc.) instead of exchange-traded index options in a systematic strategy?
The market is bigger and more liquid, hence trading costs are generally lower. In the case of swaptions you must trade OTC, the on exchange swaption market is not liquid at all.
You can get more customisation (eg of maturity dates, tenor of underlying) with an OTC transaction.
You can generally place an OTC transaction ‘with hedge’ ie you enter a spot or forward transaction at the same time as the option, using the same forward rate that’s used to value the option, such that you are delta neutral when you enter the position. With an exchange traded you need to place the deals separately, and yhere can be price slippage between the option and hedge transactions.