What is the correct order of operations when cleaning and structuring financial time series?

Question

I'm studying Lopez' Advances in Financial Machine Learning where he talks about how to sample and structure financial data, as well as how to apply machine learning models to the data. I am also following a python implementation of the book.

What isn't entirely clear to me is what is the correct order of operations to apply to financial time series for applications in machine learning. For example, in the book, the author has chapters explaining how to sample by volume rather than time, applying CUSUM filters to only label relevant sections in a time series, and applying fractional differentiation as a means of achieving memory preserving stationarity.

My question is, what is the correct order to apply the operations above? By correct I mean, what is statistically sound while also generating as many samples as possible (if we filter too much data we'll be left with a very sparse dataset).

My intuition is as follows

1. Sample tick data by volume - every time say, 1000 contracts are traded, take a sample. Lopez recommends using 1/50 of the daily average volume. The output of this sampling is a "volume bars" time series.
2. Apply feature transformation such as fractional differentiation to the volume bars time series from 1.
3. Apply event filters such as the CUSUM filter to the raw volume bars time series. The output of this will be timestamps for when we want to sample the volume bars i.e. the final samples input to train the model.

The reason behind this ordering is because if we were to swap steps 2 and 3, then the feature engineering would be applied to a much sparser dataset and would potentially miss out on information on the time series.

The author doesn't seem to explicitly state a correct ordering of operations, as the chapters don't always follow a clear timeline as to what stage to apply what operations.

Is my intuition correct? Should we apply feature engineering first before we sample the time series by events?

Answer 1

Taking a 1/50 sample seems erroneous to me.

I'm not a quant, but I did work in an algorithmic trading team. I think the problem he is trying to solve with that heuristic is called "time series simplification". You can read about how big banks solve that problem here:

Time-series simplification in kdb+: a method for dynamically shrinking Big Data Sean Keevey & Kevin Smyth, 2015.02

But what you want as your input to this polygon fitting approach is the cumulative volume curve. The fitted polygon then needs to be mapped to a wavg price.

That being said, I think there are other interesting aspects of time series data selection, many of which precede advanced mathematica/machine learning classes and are things high school students could probably do for data cleaning.

• Memory preserving stationarity - When you have two series that are clearly not stationary (the mean value is moving over time) then you can’t just regress levels against levels. At least, you need to regress changes against changes. For example, see this article on SeekingAlpha, Bitcoin: One Way to Go as an example of an incorrect linear regression plot. Testing for stationarity - Augmented Dickey Fuller (ADF) tests for the unit roots. Common statistical approaches like Arima/Box-Jenkins then eliminate causes of statistical roots, such as: Order of the autoregressive model - How many time lags are in the time series? First order differencing - A common mistake people make when comparing two time series is they compare the difference in values, rather than the rate of change. Order of the moving average model - The smoothness of the trend-cycle estimate. Fractional rate of change - Applying the above concepts, you still need to preserve the memory of the original time series. For that, ARFIMA was invented as a generalization of ARIMA.

• Lack of real world mechanism for events to naturally occur/be forced to occur - Sometimes time series analysis is done in financial publications claiming some cause-effect, but a detailed view of the transaction legs reveals there is no natural, real world mechanism for the price action to occur. And there are 1,000s of economists publishing or perishing over finding real world mechanisms for price action.

• Survivorship bias - If your symbol gets delisted, bankrupted, uplisted, halted or removed from an index or added to an index, were you aware when you constructed your study sample?

• Calendar dates can introduce sampling error - if your volume you are studying is in thinly traded issuances on the particular exchange you are studying, you need to be mindful of calendar nuances that might introduce subtle sampling bias and cause you to pick a day with very little volume, or very little composite exchange volume if the issuance trades multilaterally.

  Things to consider when selecting calendar dates:

  • sometimes the market where the security is listed doesn't observe the local holidays
  • sometimes the market does observe the holiday but the security doesn't
  • sometimes it's the other way around
  • not all securities listed on one particular market follow the same rules different market places might offer the same security but with different holiday rules
  • a lot of markets/securities have half-days too (it's like half a holiday)

• Time-resolution matters - If you think you can just sample it by date and not introduce truncation error, you are absolutely fooling yourself. If you observe a large p value, question it. Look in the news for what happened to the issuance on this date or date range in your backtest.

  • Sampling the top of every minute - Unless you are doing a closing price study, you should avoid sampling the top of every minute, especially for historical data from 2000-2010, as many naive VWAP implementations would send out order slices at the top of every minute, and could be reverse-engineered using a Fast Fourier Transform and therefore sniped by a high frequency trader who could predict order book placement and jump inside the bid/ask. - You want your sample to be truly random. The book Numerical Recipes in C has some discussion on different ways to think about randomness, although I am not an expert in how to apply such thinking to trading systems. If you do use a random number generator, you want to be able to save the seed so that you can "groundhog's day" your code.

  • "Black Swan" time series data - how normally uncorrelated time series data can become perfectly correlated, such as the price of gold and oil during market dislocations. So, in some way, I imagine you want to know what the purpose of your time series study is and correctly design your experiment around that purpose. If you are trying to find uncorrelated price movements, you probably need to make sure you handle the six sigma events separately from all the rest. But I'm not a quant.

• Special events effect error - if your random sample picks a triple/quadruple witching day, then your volume bars are not going to represent the real world very world, and so your backtest will tell you one thing, but your out-of-sample test will perform differently. Another special event is the 4.5%/48% NASDAQ 100 rebalance event.

Happy to discuss this more offline. You can find me on various social media platforms. I'd be interested to learn more as well.