Tell me more ×
Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It's 100% free, no registration required.
up vote 3 down vote favorite

Say for a stock I want to do a simulation using 30 days of historical returns, and maybe generate 1000 paths, with 2 days as the forecast horizon. Say I have 100 of these 5 day blocks used for generating the distribution, matched with the actual values of the 2 days I am interested in forecasting. How would I estimate the accuracy of this method of generating a distribution? And what is a method I could use to compare it with a different means of forecasting the distribution? Any suggestions would help.

share|improve this question
I haven't got it. How would you like to do a simulation if you have historical data? – Ilya Feb 22 '11 at 13:34
one way to do it is generating sample paths based on some stochastic process. the inputs are usually the riskless rate and an estimation of volatility. – user468 Feb 23 '11 at 19:27

3 Answers

up vote 1 down vote

Compare the density and cumulative density function or your forecast volatility and of the realized volatilityenter image description here

share|improve this answer
the problem i'm having, and probably i am not thinking about this the right way, is that when estimating a distribution, if the forecast differs significantly from realized, how do i know the forecasted distribution is off, if it could just be the realization of some upper or lower quartile of the distribution. since i'm not making a point forecast, it seems difficult to tell. – user468 Feb 23 '11 at 19:27
the idea is to take your model and generate many forecast, comparing the densities of those with the observed realizations it should pop out if your model does an adequate job (see graph). You may also may also want to explore MCMC methods in order to get a better understanding of the studied data before trying to forecast it. – phil Feb 24 '11 at 23:53
I was thinking along similar lines, that the trick was to do many many simulations and then combine them all at the end. I was first having a hangup about how to combine them all but I figured just placing them in a distribution(the realized)based on their forecasted probability was good enough. Then measuring any errors. Thank you for your answers – user468 Mar 15 '11 at 20:01
up vote 1 down vote

Seems like you are interested in the forecast error

http://en.wikipedia.org/wiki/Forecasting

With regard to distribution, I would also look at the mean forecast error. A good model with have a mean error = 0, since it is not bias upward or downward.

-Ralph Winters

share|improve this answer
up vote 0 down vote

You might want to perform a Two sample Kolmogorov-Smirnov test on the empirical CDFs of the distributions: your forecast and the realized distribution. Then use the p-value of the test as the metric of interest. Other statistical tests of difference in distribution which can be so abused are the Baumgartner, Weiss, Schindler test, and the variations thereupon. I believe there is also a 2-sample Anderson-Darling type test. There are also the non-symmetric Kullback Leibler divergence, and a half-dozen other definitions of statistical distance. Probably you are most limited by the statistical package or programming language you are working with. If using R, I would guess you can find implementations of all of the above.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.