# Assess forecasting performance of model in presence of bid-ask bounce

I have a forecasting model for 1-minute asset returns $$y_t$$ derived from trade data. (The assets are not very liquid.) The predictors of the model include the lagged target variable $$y_{t-1}$$, which also takes one of the top spots in the model's variable importance table. The partial dependence plot of $$y_{t-1}$$ shows a pronounced negative slope. My understanding is that this effect is most likely due to the bid-ask bounce, but I am not sure because I do not have a lot of experience with market microstructure. Since I am stuck with trade data at the moment, I wanted to assess to what degree the bid-ask bounce is responsible for my model's performance, or -to be more precise- to check whether my model has some predictive power left after accounting for the bid-ask bounce.

My idea is as follows:

1. Generate normally-distributed synthetic mid price returns with a volatility derived from real data
2. Add a bid-ask bounce from a very "strong" bid-ask bounce process on top of the mid price - I was thinking of a time series alternating between -BASpread/2 and +BASpread/2 (using the mean BASpread derived from real data)
3. Train an AR(1) model on the returns of the synthetic data (with BA bounce included).
4. Record the AR(1) model's out-of-sample performance on new synthetic data (same parameters as in points 1 & 2). By definition the AR(1) model will do really well in modeling the bid-ask bounce. Therefore, this should give me some sort of "benchmark" that I can use to compare my model against.

Repeat steps 1-4 for, let's say, 100 times and calculate the mean out-of-sample performance of the AR(1) models. Finally, compare the AR(1) performance to my model's performance. The assessment of whether my model has a "significantly" better performance than the AR(1) model is worth another question, I guess. For now, I hope that the outcome of this experiment is clear enough so that I can make a gut-feeling decision.

Questions

1. Do you think this is a valid approach or do I miss something? If so, please elaborate.
2. Given the limitations of my data, can I do better than the above in order to see whether my model has predictive power after accounting for the bid-ask bounce?
• Hi. Out of curiosity, how did you clean your trade data? And does your model use 1-min asset returns to forecast 1-minute ahead or does it forecast longer periods? While the bid-ask bounce contributes to microstructure noise, there will be other factors that will contribute as-well (eg. discreteness of the asset price, see this paper). Instead of only considering noise created by bid-ask bounce, why not just consider a general noise-term and see how your forecast model fares in this framework?
– Pleb
Jun 5 at 8:41
• The data looks quite well behaved and I did not apply any cleaning. Yes, I forecast 1 minute ahead and also use the past 1 minute return as predictor (among other things). Thanks, for the link to the paper about the discreteness of asset returns - this might indeed contribute to the negative autocorrelation that I see. However, the tick sizes of the assets that I use for model training are relatively small - so the bid ask-bounce (or other effects, as you write) might have more influence. Jun 5 at 12:24
• Okay, I suspect that your data has been pre-cleaned, if it is well-behaved. Where did you get the intraday data? and are you modeling returns directly or do you model intraday return variation (ala. GARCH) and then back out returns from this model? In general, I would still consider creating a general noise-term and observe how the forecasting model fares. If it performs poorly in this scenario, then it will likely also perform poorly under the presence of bid-ask bounce. This follows the same structure as you suggested above, however at (2) simulate a noise-term following eg. $N(0,1)$.
– Pleb
Jun 5 at 18:17
• Yes, the data might be pre-cleaned - I get it directly from the exchange. The returns are modeled directly. It's still a very basic model but I do get quite good results, which is why I have been suspecting that the bid-ask bounce is responsible for it. I have also been thinking about using a normally-distributed noise term in step (2) - actually this was my initial idea. However, the AR(1) benchmark model will do worse on this data, i.e., it will produce a lower benchmark performance that my "real" model can beat more easily. That was the reason why I went for the +/- alternating process. Jun 7 at 6:17