My question contains many parts, but I will try to keep it somewhat focused. I am primarily looking for a framework to evaluate the accuracy of a stock-focused Options Pricing Model. One of the hardest questions seems to be determining what the right answer is, even in hindsight.

Most common pricing models seem to work on the basic framework established by Black, Scholes, and Merton. There is an interest rate component, some base probability distribution and a way to relate those to time and underlying price changes. Each model gives roughly similar answers, but can differ in many details.

I am looking for generally accepted approaches to evaluate the output of these models vs reality. My goal is to be able to run the models on a moderately large batch of stocks (~top 300 by options liquidity) for the last 15 years and generate quantitative measurements about their effectiveness.

These measurements would allow me to rank the models relative to each other. This would be compared to market factors to determine the overall quality of the model and how its quality changes in particular market conditions.

At this point, my main focus is comparing the Implied Volatility output of the function to the future reality. My assumption, and I am specifically asking for feedback on this, is that an OPM with a lower Implied Volatility vs Realized Volatility error measurement would be considered a more accurate model.

This would not be used to rate the OPM directly. I am honestly not sure what the "Correct" error rate would be or what that would mean. The goal is to compare error rates between different models side by side over the same data set. This should allow me to determine which is the comparatively better model based on its ranking of error.

I am following the basic theory that all options prices represent a pricing of future risk by the market. The information traders use to determine risk is incomplete and therefore contains a natural (but possibly varying) level of prediction error. Option Pricing Models induce an additional level of error based on their underlying assumptions. A "Better" model would be one that induces a lower level of error vs the realized volatility.

So here are the specifics;

  • Does comparing Implied Volatility to Realized Volatility make sense as a quantitative measurement of an OPM's effectiveness?
  • Is there anything fundamentally wrong with this concept I should be aware of?
  • Are there any generally accepted methodologies for doing this analysis?
  • Are there other measurements of an OPM's effectiveness that are better or should be considered?
  • What comparisons would make the results of this analysis useful for model selection? (ie vs Interest Rates, vs Historic Volatility, vs Market Shocks, vs Time to Expiration, vs strike distance, etc...)
  • What OPMs should be included in the analysis? (Taking nominations and I will share results.)

I would appreciate constructive comments on the overall concept, but analysis approaches, previous papers and tool suggestions would be more helpful. I am using R, MATLAB and C# (or .NET in general) as my toolkit for doing the work. Any pre-existing tools that would work with those would be extremely appreciated.

The computational efficiency of the model isn't as relevant in this situation, but I may want to compare that at a different time.

If the results are meaningful in any way I am happy to share them and I would welcome collaboration if anyone is interested (make a note in the comments.)

  • 1
    $\begingroup$ Interesting question. I would not compare implied vs realised volatility to judge of the effectiveness of a pricing model though. I would do something more along the lines of what Rebonato discusses in this very nice paper: papers.ssrn.com/sol3/papers.cfm?abstract_id=2365294. That is, determine whether when one sells an option at its fair price (i.e. theoretical or model price) + margin (business), he/she can indeed use the fair price to replicate the position and deliver the client exactly what he/she asked for in terms of cash flow, while conserving the margin. $\endgroup$
    – Quantuple
    Commented Aug 27, 2016 at 22:31
  • $\begingroup$ @Quantuple I am not sure if I made this clear. I am not looking at the error rate of Implied Volatility vs Realized Volatility to determine an absolute value of effectiveness for a specific OPM. What I am doing is looking at error rates of multiple OPMs over similar samples and using the error rate to compare relative effectiveness between the different OPMs. $\endgroup$
    – drobertson
    Commented Aug 28, 2016 at 0:41
  • $\begingroup$ @Quantuple I have no idea what an IV vs RV error rate would mean individually, but I do know that a better OPM should, in general, have a lower error rate than a poor model. This is based on the idea that the OPM model induces some level of model error beyond the natural market error due to uncertainty. The goal is to find the model that induces the least relative error. $\endgroup$
    – drobertson
    Commented Aug 28, 2016 at 0:47
  • $\begingroup$ @Quantuple Thanks for the reference to the paper. An interesting read. $\endgroup$
    – drobertson
    Commented Aug 28, 2016 at 5:28
  • $\begingroup$ Yes I understand. But I meant that a model embeds more than a view of future volatility. Focusing on volatility might be too restrictive in sole cases. More specifically, two diffusion models could perfectly agree on the future realised volatility, but not on the smile dynamics for instance (see LV vs VS calibrated to the same vanilla market, no jumps). $\endgroup$
    – Quantuple
    Commented Aug 28, 2016 at 8:56


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.