Say for a stock I want to do a simulation using 30 days of historical returns, and maybe generate 1000 paths, with 2 days as the forecast horizon. Say I have 100 of these 5 day blocks used for generating the distribution, matched with the actual values of the 2 days I am interested in forecasting. How would I estimate the accuracy of this method of generating a distribution? And what is a method I could use to compare it with a different means of forecasting the distribution? Any suggestions would help.
$\begingroup$
$\endgroup$
2
-
$\begingroup$ I haven't got it. How would you like to do a simulation if you have historical data? $\endgroup$– SBFFeb 22, 2011 at 13:34
-
$\begingroup$ one way to do it is generating sample paths based on some stochastic process. the inputs are usually the riskless rate and an estimation of volatility. $\endgroup$– user468Feb 23, 2011 at 19:27
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
Seems like you are interested in the forecast error
http://en.wikipedia.org/wiki/Forecasting
With regard to distribution, I would also look at the mean forecast error. A good model with have a mean error = 0, since it is not bias upward or downward.
-Ralph Winters
$\begingroup$
$\endgroup$
3
Compare the density and cumulative density function or your forecast volatility and of the realized volatility
-
$\begingroup$ the problem i'm having, and probably i am not thinking about this the right way, is that when estimating a distribution, if the forecast differs significantly from realized, how do i know the forecasted distribution is off, if it could just be the realization of some upper or lower quartile of the distribution. since i'm not making a point forecast, it seems difficult to tell. $\endgroup$– user468Feb 23, 2011 at 19:27
-
$\begingroup$ the idea is to take your model and generate many forecast, comparing the densities of those with the observed realizations it should pop out if your model does an adequate job (see graph). You may also may also want to explore MCMC methods in order to get a better understanding of the studied data before trying to forecast it. $\endgroup$– philFeb 24, 2011 at 23:53
-
$\begingroup$ I was thinking along similar lines, that the trick was to do many many simulations and then combine them all at the end. I was first having a hangup about how to combine them all but I figured just placing them in a distribution(the realized)based on their forecasted probability was good enough. Then measuring any errors. Thank you for your answers $\endgroup$– user468Mar 15, 2011 at 20:01
$\begingroup$
$\endgroup$
You might want to perform a Two sample Kolmogorov-Smirnov test on the empirical CDFs of the distributions: your forecast and the realized distribution. Then use the p-value of the test as the metric of interest. Other statistical tests of difference in distribution which can be so abused are the Baumgartner, Weiss, Schindler test, and the variations thereupon. I believe there is also a 2-sample Anderson-Darling type test. There are also the non-symmetric Kullback Leibler divergence, and a half-dozen other definitions of statistical distance. Probably you are most limited by the statistical package or programming language you are working with. If using R, I would guess you can find implementations of all of the above.
answered Mar 3, 2011 at 3:38