Market data and machine learning

Question

I have the following general question regarding the use of ML in quantitative finance:

Lets say I want to train a model (for simplicity lets consider a neural network), so that I feed some market data for a stock, and I want to compute something on it (the specific goal is not important here, imagine just a pricer). What happens is that usually the market data on a stock has a non-fixed structure. For example for stock $A$ I might have $n$ dividends, whereas for stock $B$ I may have $m$. Therefore, feeding dividend data into the model is not a straight-forward task, as the size of the entry values for the model is not constant and the input for a NN has to be an array of fixed size. I used the dividends example, but this applies to everything: volatility surface (I might have a $n_1 \times n_2$ matrix for one stock and a $m_1 \times m_2$ sized volatility surface for another), etc.

For rate curves PCA decomposition is well suited, so it might be simple to transform a variable-sized vector of date-value tuples into a fixed $d$-dimensional structure. But what about other inputs? What is the common practice?

I know there are many possibilities in ML, but I would like to know what are the ones that are more readily or well suited to be used in finance. References are also welcome.

Answer 1

LSTM and Padding are useful approaches!

I'm sharing some links below for your reference.

The first one gives a quick glimpse into padding techniques for time series data, using Python (1D CNN).

The second and third ones talk about handling data with variable input sizes.

The last two dive into the usage of LSTM for time series forecasting.

1. https://medium.com/full-metal-data-scientist/an-introduction-to-time-series-padding-techniques-in-python-b7307a2eba87

2. https://stackoverflow.com/questions/38189070/how-do-i-create-a-variable-length-input-lstm-in-keras

3. https://datascience.stackexchange.com/questions/48796/how-to-feed-lstm-with-different-input-array-sizes

4. https://machinelearningmastery.com/lstm-for-time-series-prediction-in-pytorch/

5. https://blog.quantinsti.com/rnn-lstm-gru-trading/

Hope this helps!

(Disclaimer: I'm associated with Quantinsti, the platform for the 5th link).