# How to interpret the accuracy result of the forecaste?

I'm trying to forecast the vacancy rate of multifamily rental property. I have the data from 1992 until today. I'm trying to fit a model with the serie without the last 2 observations.

I only need forecasting for 2 years into the future.

My question is : Which model is better in forecasting?

The first model seem to do a better job with the training set than the second model. But the reality seem to be reversed when we look at the test set of only 2 periods.

I think I should look at the test set, but at the same time, I wonder if it could be only better by chance since the test set is comprised of only 2 observations.

Should I look at the training set or at the test set? Which model will you choose and why?

Thank you in advance!

I'm using R.

accuracy(fit1, ti_canada[26:27])

                  ME      RMSE       MAE        MPE      MAPE      MASE       ACF1


Training set 0.00944111 1.1486731 0.8727625 -13.108775 28.172474 0.7247854 -0.0850986

Test set -0.19295099 0.5297774 0.4933903 -3.432013 8.200891 0.4097359 NA

accuracy(b6, ti_canada[26:27])

                  ME      RMSE       MAE       MPE      MAPE      MASE     ACF1


Training set 0.09192687 1.3890104 1.1561171 -9.349635 39.581546 0.9600973 0.387796

Test set -0.31354649 0.3881324 0.3135465 -5.268731 5.268731 0.2603846 NA

## 1 Answer

The question of choosing the best model, is a matter of balance.

All your indicators (ME, RMSE, MAE, MPE, MAPE, MASE, ACF1,...) are aggregations of two types of errors : a bias (you have the wrong model but an accurate fit) + a variance (you have the right model but a inaccurate fit). And there is no statistical method to know if you have a high bias and low variance or a high variance and low bias.

So I suggest, you make a plot and make an eye-stimate to select the "best" one, best meaning with the least business consequences if you are wrong.

Furthermore, you should rejoice. Having two forecast gives you an idea of how you can be wrong. Because, face it, you will be wrong! What maters is not to be very far off.

Knowing in advance the precision may be reassuring. But on the bottom line, where would you loose more money: by underestimating or overestimating the vacancy rate ?