I'm studying Lopez' Advances in Financial Machine Learning where he talks about how to sample and structure financial data, as well as how to apply machine learning models to the data. I am also following a python implementation of the book.

What isn't entirely clear to me is what is the correct order of operations to apply to financial time series for applications in machine learning. For example, in the book, the author has chapters explaining how to sample by volume rather than time, applying CUSUM filters to only label relevant sections in a time series, and applying fractional differentiation as a means of achieving memory preserving stationarity.

My question is, what is the correct order to apply the operations above? By correct I mean, what is statistically sound while also generating as many samples as possible (if we filter too much data we'll be left with a very sparse dataset).

My intuition is as follows

  1. Sample tick data by volume - every time say, 1000 contracts are traded, take a sample. Lopez recommends using 1/50 of the daily average volume. The output of this sampling is a "volume bars" time series.
  2. Apply feature transformation such as fractional differentiation to the volume bars time series from 1.
  3. Apply event filters such as the CUSUM filter to the raw volume bars time series. The output of this will be timestamps for when we want to sample the volume bars i.e. the final samples input to train the model.

The reason behind this ordering is because if we were to swap steps 2 and 3, then the feature engineering would be applied to a much sparser dataset and would potentially miss out on information on the time series.

The author doesn't seem to explicitly state a correct ordering of operations, as the chapters don't always follow a clear timeline as to what stage to apply what operations.

Is my intuition correct? Should we apply feature engineering first before we sample the time series by events?

  • $\begingroup$ This is a good question, the AdvFinML book talks about a lot of operations, but it doesn't mention the overall pipeline to link these operations together $\endgroup$ – soulmachine Jun 9 '20 at 9:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.