Questions tagged [validation]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
9 votes
2 answers
673 views

Risk Model Validation

I have such a general question regarding risk model validation. Which tools are most often used for validation and how does the process work? Could you recommend any books that focus on this topic?
user avatar
  • 413
0 votes
0 answers
234 views

Train/test: why 80:20 split performed better than 90:10 split?

Playing with Random Forest Classifier, I am wondering what could cause in a 80:20 split the test results to perform better than in a 90:10 split? With 2000+ data points and: with 80:20 split, ...
user avatar
  • 1
1 vote
0 answers
93 views

Multivariate combinatorial purged cross-validation

Combinatorial purged cross-validation (CPCV) is a technique for backtesting strategies while purging and embargoing observations in a time series. CPCV improves upon classical k-fold and walk-forward ...
user avatar
  • 2,815
0 votes
1 answer
133 views

Sampling and cross-validating with tick, volume and dollar bars

Financial data is usually structured with time bars. Other sampling techniques include: tick bars volume bars dollar bars. These are so-called sampling techniques to better identify signals and ...
user avatar
  • 2,815
2 votes
0 answers
79 views

The discontinuity when applying the combinatorial purged cross-validation

In Marcos Lopez de Prado's book, Advances in financial machine learning, he recommends using the combinatorial purged cross-validation(CPCV) for backtesting. His motivation is sensible. Through the ...
user avatar
1 vote
0 answers
32 views

Is C(P)CV similar to boostrap?

I am writing to ask if the Combinatorial (Purged) Cross-Validation" method of Marcos Lopez de Prado's "Advances in Financial Machine Learning" book is similar to the idea of bootstrap. If not, what is ...
user avatar
  • 31
2 votes
0 answers
153 views

LGD performing model - LGD estimate vs LGD observed

LGD (Loss Given Default) performing model is developed on through the cycle sample which consists of loans in default. What I want is to compare LGD estimate and LGD observed (realized). LGD observed ...
user avatar
1 vote
0 answers
296 views

Berkowitz test for CVaR backtesting

I want to test CVaR using the Berkowitz test (focus on the left tail). I have a couple of doubts: Do I need to transform only actual losses that are above CVaR; In the first transformation, whether ...
user avatar
1 vote
0 answers
141 views

PD validation in the low/no default setting

The topic of this question is the validation as prescribed in the Basel N ($N \ge 2$) framework. The task is given the probability of default $p_k$ for $K$ rating classes at time $t$ and the outcome ...
user avatar
  • 13.2k
1 vote
0 answers
219 views

Testing whether a process is a Wiener process [closed]

Ideally I would like links to code implementations (eg. Matlab ) or book/paper references, but I would appreciate suggestions on various methods. Update: I was hoping to attract people who test the ...
user avatar
2 votes
2 answers
483 views

ES not elicitable

Expected Shortfall is not elicitable as some papers have pointed out. That simply means that there is no scoring function that elicits ES. My question is, does this imply that Expected Shortfall ...
user avatar
  • 43
1 vote
1 answer
318 views

Kfold cross validation: how to handle hold-out periods

I want to backtest a strategy using K-fold cross validation. Assume I have a period of 300 days in my backtest. I divide it into 30 folds of 10 days each. On day 1 of a fold, I enter a trade, and ...
user avatar
13 votes
3 answers
1k views

Validating a Credit Scoring Model without Data

Fellow Quants, Suppose you have a credit scoring model that is developed without the aid of statistics, because (unfortunately) there is no historical default/loss data in your portfolio. The ...
user avatar
  • 386
0 votes
0 answers
474 views

Examples for the option model validation

When implementing a code for the new model, even if it provides sensible price, it is still a good idea to compare it against some benchmarks, even in the special case of constant volatility Black-...
user avatar
  • 1,474
1 vote
1 answer
537 views

Validation of Bates SVJ model

I have just finished implementing the Bates model for pricing European call options. To check results, I have been looking for a validation set where I could see the Bates parameter values and ...
user avatar
  • 1,391