Take the 2-minute tour ×
Quantitative Finance Stack Exchange is a question and answer site for finance professionals and academics. It's 100% free, no registration required.

If a random variable is discrete, and we are interested in its quantile value, how to define a proper back testing procedure?

For example, the underlying variable with a discrete value is

$$ d(\mbox{account}) = \mbox{PaymentDate} - \mbox{BillingDate} $$

the observing variable:

$$ y = \mbox{percentile}(d, 95\%, \mbox{month}) $$

or $y$ is the 95th percentile value of $d$, for a particular month. e.g. 95% of credit cards are paid within 20 days from the billing, in 2013 Jan.

How could I define a back-testing approach?


Background

To define an estimation-backtesting method for a continous random variable is easier. Now in my group we have such a non-parametric approach:

underlying variable:

$$r(\mbox{month}) = \mbox{monthly credit-card account default rate}$$

For example, 2013 Feb default rate is 1.1%, 2013 Jan is 1.2%...

observing variable:

$$ x = \mbox{percentile}(r, 95\%) $$

$x$ is the 95% percentile value of $r$. Here $x$ definition is similar to VaR.

point forcast:

$$ \hat x(\mbox{month}) = \mbox{percentile}(r(\mbox{month}), N, 95\%) $$

$\hat x$ is the 95% percentile value of $r$, based on $N$ historic observations.

For example, take $N=36$, retrieve back 36 months, the 95% percentile value of default rate $r$ is 2.3%. then $\hat x = 2.3\%$.

point forecast Exception:

$$ \mbox{PFException}(t) = \begin{cases} 0 & r(\mbox{month}) \leq \hat x(\mbox{month}) \\ 1 & \text{otherwise} \end{cases} $$

By right 95% of the time there shall have no exception, while 5% of the time exception happens.

backtesting:

There are POF test, checking the rate of the exception; and independent test, checking the correlation of exceptions.

For example, Kupiec (1995) proposed a POF test checks exceptions happened in previouis 36 months' point forecasts: 0-4 exceptions are ok, green light, 4-7 exceptions are yellow light, while more than 8 exceptions are red light.

Christoffersen (1998) proposed an independent test.

Kupiec, P. (1995). Techniques for verifying the accuracy of risk management models. Journal of Derivatives 3, 73–84.

Christoffersen, P. (1998). Evaluating interval forecasts. International Economic Review 39, 841–62.

share|improve this question
3  
I've done some heavy surgery to your $\LaTeX$ to make everything readable. Kindly use this scheme from now on. –  chrisaycock Mar 11 '13 at 14:35
    
@chrisaycock thanks a lot! –  athos Mar 11 '13 at 16:16

1 Answer 1

up vote 1 down vote accepted

Just to answer my own question. Discrete variates' quantile, could be modelled and tested. I followed this paper: "Discrete Quantile Estimation", Halina Frydman and Gary Simon, New York University, January 29, 2007. Detailed approach is inside.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.