5 votes
Accepted

how to I get the statistical significance of a backtested result

Compare Sharpe , Sortino Ratios, yearly Profit,Max Drawdowns per year of your strategy to 1) buy and hold all of the stocks in your universe 2) few strategies (with different random seeds) which ...
user avatar
  • 796
4 votes

Variable becomes more significant when more variables are included

A Change in the statistical significance of a variable when more variables are included can come from two sources. One is the fact that the estimate of the coefficient becomes larger when new ...
user avatar
  • 650
3 votes

Evaluate the significance of the relationship among VIX and the S&P 500

It is unlikely with so few observations that you will get statistical significance. Also you should benchmark your model against a "naive" model. If your conjecture is that high VIX implies high ...
user avatar
  • 6,860
2 votes

R squared statistic in predictions of returns

The goal of regression is to account for the variance in $y$. If you are able to do that, then your predictions of the conditional mean of $y$ (conditioned on the values of your features) will be ...
user avatar
  • 161
2 votes

The use of $p$-value in finance after the recent statement of ASA (American Statistical Association)

Deidre McCloskey has been going on about this for as long as I can remember. See for example the aptly titled : "The cult of statistical significance: How the standard error costs us jobs, justice and ...
user avatar
  • 4,217
2 votes

Does historical backtest data mean anything?

There's a field of study called Statistics, which to a large extent tries to answer questions like that both in a financial setting and in experimental sciences. Try to read something about it. To ...
user avatar
  • 1,553
2 votes
Accepted

Does historical backtest data mean anything?

If your model is only relating to historical price data of that single stock, then the model wouldn’t be useful. Historical price data is stochastic, and a lot of theory in financial mathematics is ...
user avatar
2 votes

Does the default rate follow normal distribution or binomial distribution?

Normal rv can take negative values so won’t work for default rate (which is positive) without some form of transformation- classic approach is Vasicek, which by making assumptions about the default ...
user avatar
2 votes
Accepted

N Needed for Statistically significant tracking error

Suppose you have a portfolio that has some unknown real mean tracking error, $t$, relative to the benchmark, with some real variance $\sigma_t^2$. You have a sampling process generated from your ...
user avatar
  • 8,047
2 votes

CAPM alphas have unexpected p-value distribution

The distribution of p-values is uniform only when the null hypothesis is true and all assumptions are met. The difficulty with p-values is that they are composite functions. Finding that an empirical ...
user avatar
  • 4,092
1 vote

How to test the difference between samples of sharpe ratios

Typically we have $$\hat{\zeta}\approx\mathcal{N}\left(\zeta,\frac{1 + \frac{\zeta^2}{2}}{n}\right),$$ where $\hat{\zeta}$ is the observed Sharpe ratio, and $\zeta$ is the unobserved population ...
user avatar
  • 2,806
1 vote

question about significance level

I figured it out now. When calculate the probability use Binominal itself, the result matches: $$\alpha=1-(P(4)+P(3)+P(2)+P(1)+P(0))=1-(0.134+0.214+0.257+0.205+0.081)=0.108$$
user avatar
  • 213
1 vote

Pearson correlation significance : Issue with $t$-statistic increasing with $N$

The question of significance is not about the correlation but about the precision of the estimation. If the value estimated with more data is still near the same value estimated with less data, that ...
user avatar
  • 2,750
1 vote

Should Fama-French coefficients be calculated with daily or monthly returns?

In this context, I refer to the book on Empirical Asset Pricing by Bali, Engle and Murray (2016). They state on page 124 that A minimum number of data points are usually required to ensure the ...
user avatar
  • 61
1 vote
Accepted

Test statistic of event study

If I understand it rightly, I have to take the mean of the abnormal returns for the 50 companes at $t_0$ as the numerator. for example: $\bar A_{t} \ = \ \frac{1}{N_t}\sum_{i=1}^{n_t}A_{i,t'} $ ...
user avatar
  • 2,936
1 vote

Average Return Differential Calculation - Newey West t-Statistic

The values for $VaR$ and $CF-VaR$ are the time-series means for each cross-sectional values. As mentioned in the table description, you calculate both variables for each month from Jan. 1995 to Dec. ...
user avatar
  • 2,936
1 vote

Assessing goodness of a Technical Trading Rule using a ROC model

So ROC performance metrics are common place in machine learning applications, particularly for classification tasks. Looking at the way you have setup the question here you wont be able to plot a ...
user avatar
1 vote

Kolmogorov-Smirnov test for Generalized Pareto Distribution

As an additional (simple) solution I would use the probability integral transform (PIT) of the returns with respect to the generalized pareto distribution. Under the null hypothesis that the ...
user avatar
  • 2,514
1 vote

Kolmogorov-Smirnov test for Generalized Pareto Distribution

I don't know if there are any additional issues that arise with using goodness off fit with a piece-wise function. When I have fit generalized pareto distributions to series like financial market ...
user avatar

Only top scored, non community-wiki answers of a minimum length are eligible