2
$\begingroup$

what is the best format to feed the input data, which are time series with varying density over time, to a deep learning network, while at any iteration we want to feed a batch of data including a historical background?

Is it better to consider a constant size of data records or a constant time window including variable data record size? Or is there a better way?

$\endgroup$
2
$\begingroup$

I am not sure I perfectly understand your question, the concept of "time series with varying density over time" is not very clear.

One thing is for sure, the optimal way to "feed" a neural network is a function of the type of NNet itself and of the learning method you have chosen.

For time series

  • either you believe your data are iid vectors, and you can use a fully connected perceptron
  • either they are auto-correlated, and probably an LSTM (Long-Short Time Memory) is a good option to consider
  • if you expect them to have multi-scale patterns ConvNets are probably a good choice.

For the learning, the way you feed the NNet has to be compatible with your previous choice:

  • fully connected perceptrons on iid observations should be trained via Stochastic Gradient Descent (SGD), ie using random small batches
  • LSTM have to be feed by long blocks of consecutive data,
  • ConvNets have to be feed by blocks preserving the structure of your expected patterns.

Here are few remarks about your last point about using a fixed window in number of observations versus an observation duration:

  • you need to have a decent number of observations in each batch, hence if you choose a duration, take it long enough
  • you need to not break the type of pattern you expect the NNet to capture; take your duration (or number of points) long enough.

In essence, for mini-batches driving a SGD, their composition has to be chosen such that the estimated gradient (stemming from the averaging over the mini-batch), has the same variance from a mini-batch to the other. Look at your data and make the best choice!

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.