Tell me more ×
Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It's 100% free, no registration required.
up vote 10 down vote favorite
4

Let's say I have a brand new fancy model on some asset class (calibration porcedure included over a set of vanilla options) in which I truly believe I made a step forward comparing to existing litterature (a phantasmatic situation I recognize).

Which criteria should be in order so I can claim that I am in a position to "validate" this model ?

Please open fire

Regards

PS : We will assume that the math are OK and that I can prove that there is no bug in the model implementation

share|improve this question
2  
I am not a professional quant but I am a professional simulation modeler. I can tell you that a validation process is something that is agreed upon by all stakeholders, as there are almost infinitely many metrics and aspects of any non-trivial model that may be examined. In a nutshell, it's valid when you say it's valid, according to your standards. – Pete Feb 4 '11 at 16:16

1 Answer

up vote 5 down vote

I don't think that there is a precise point in time when we can say that model is valid (well, it's a model not a law). For example, E. Derman in his article on Model risk describes the verification of model as a iterative process:

It is impossible to avoid errors during model development, especially when they are created under trading floor duress .... So, after the model is built, the developer tests it extensively. Thereafter, other developers “play” with it too. Next, traders who depend on the model for pricing and hedging use it. Finally, it’s released to salespeople. After a suitably long period during which most wrinkles are ironed out, it’s given to appropriate clients. This slow diffusion helps eliminate many risks, slowly but steadily.

He also describes 6 types possibilities which constitutes model risk here:

  • Incorrect model
  • Correct model, incorrect solution
  • Correct model, inappropriate use
  • Badly approximated solution
  • Software and hardware bugs
  • Unstable data

In addition to that he provides some tips that can be used to avoid model risk and its consequences (to name just few):

  • Test complex models in simple cases first
  • Test the model’s boundaries
  • Don’t ignore small discrepancies
share|improve this answer
We view validation and verification as two separate processes. "Verification": does the program and model do what it is supposed to/designed to do? E.g. if it is supposed to trade when condition x is true, does it? "Validation": does it predict well enough to make money. – Pete Feb 5 '11 at 18:48
Well I know this paper and I find it really unsatisfactory. Anyway it is principally about operational risk in implementing a model (case that I discarded in my question) rather than model validation in my opinion. Anyway when you've finished reading it you still don't realy know what to do exactly with your new model to know how it can be validated. – TheBridge Feb 5 '11 at 21:03

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.