4
$\begingroup$

If a random variable is discrete, and we are interested in its quantile value, how to define a proper back testing procedure?

For example, the underlying variable with a discrete value is

$$ d(\mbox{account}) = \mbox{PaymentDate} - \mbox{BillingDate} $$

the observing variable:

$$ y = \mbox{percentile}(d, 95\%, \mbox{month}) $$

or $y$ is the 95th percentile value of $d$, for a particular month. e.g. 95% of credit cards are paid within 20 days from the billing, in 2013 Jan.

How could I define a back-testing approach?


Background

To define an estimation-backtesting method for a continous random variable is easier. Now in my group we have such a non-parametric approach:

underlying variable:

$$r(\mbox{month}) = \mbox{monthly credit-card account default rate}$$

For example, 2013 Feb default rate is 1.1%, 2013 Jan is 1.2%...

observing variable:

$$ x = \mbox{percentile}(r, 95\%) $$

$x$ is the 95% percentile value of $r$. Here $x$ definition is similar to VaR.

point forcast:

$$ \hat x(\mbox{month}) = \mbox{percentile}(r(\mbox{month}), N, 95\%) $$

$\hat x$ is the 95% percentile value of $r$, based on $N$ historic observations.

For example, take $N=36$, retrieve back 36 months, the 95% percentile value of default rate $r$ is 2.3%. then $\hat x = 2.3\%$.

point forecast Exception:

$$ \mbox{PFException}(t) = \begin{cases} 0 & r(\mbox{month}) \leq \hat x(\mbox{month}) \\ 1 & \text{otherwise} \end{cases} $$

By right 95% of the time there shall have no exception, while 5% of the time exception happens.

backtesting:

There are POF test, checking the rate of the exception; and independent test, checking the correlation of exceptions.

For example, Kupiec (1995) proposed a POF test checks exceptions happened in previouis 36 months' point forecasts: 0-4 exceptions are ok, green light, 4-7 exceptions are yellow light, while more than 8 exceptions are red light.

Christoffersen (1998) proposed an independent test.

Kupiec, P. (1995). Techniques for verifying the accuracy of risk management models. Journal of Derivatives 3, 73–84.

Christoffersen, P. (1998). Evaluating interval forecasts. International Economic Review 39, 841–62.

$\endgroup$
  • 3
    $\begingroup$ I've done some heavy surgery to your $\LaTeX$ to make everything readable. Kindly use this scheme from now on. $\endgroup$ – chrisaycock Mar 11 '13 at 14:35
1
$\begingroup$

Just to answer my own question. Discrete variates' quantile, could be modelled and tested. I followed this paper: "Discrete Quantile Estimation", Halina Frydman and Gary Simon, New York University, January 29, 2007. Detailed approach is inside.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.