I have some 1-minute bar data. The first datapoint has time t0 and the last one t1. 99.5% of the data between the first timestamp and the last timestamp is there and 0.5% is missing (NaN values). I can confirm that there is a data point for every single minute between t0 and t1.

As a data pre-processing step, for consistency, rather than dropping the NaN values, I was wondering if I can substitute them with a rolling mean or median. Because there are very few missing values (0.5%) this shouldn't cause any ill effects. Would you do this? Would you use the median or mean or just forward/front fill or back fill?

  • $\begingroup$ OK, but rather than use a median or mean I would use the "last known value", since in a financial application it is the best estimate of the current unknown value (there is roughly equal chance that the price went up or down since your most recent observation). $\endgroup$ – noob2 Aug 15 at 17:11
  • $\begingroup$ Thanks - I'll forward fill then. However, when I take the return series, I'll end up with more than usual 0 return periods - I guess this is OK. $\endgroup$ – s5s Aug 15 at 17:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.