Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.

# Tag Info

0

I have actually considered the problem that you are working on, though configured somewhat differently. There isn't going to be a universal answer to your question. See, in particular, Holland, Paul W. Covariance Stabilizing Transformations. Ann. Statist. 1 (1973), no. 1, 84--92. Nonetheless, there are answers, some already mentioned. I would argue ...

0

For each of the examples provided in the question, if you had some view as to how X changes Y, you would update your $\mu$, not your $\Sigma$. Your covariance matrix by definition is meant to measure RANDOM noise between two variates. Your sports examples are just as applicable to stocks. In the tech industry, company A manufactures parts overseas, but ...

1

This is not a complete answer, more a different perspective to the answers already given. If you have some a-priori knowledge about the covariance structure and about the factors influencing it, you should try to reflect this in your statistical model. Three ideas: Divide your sample into subpopulations with identical factor values and estimate separately. ...

2

As an addition to the already rich answers, I would suggest you to read the following paper by Marcos L. De Prado on the computation of Forward-Looking Correlation Matrices. Estimation of Theory-Implied Correlation Matrices https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3484152

2

The Ledoit-Wolf estimate cited by @develarist can be quite good, but as you say you already knew about "shrinking". It takes the population of correlations observed as an effective Bayesian prior for any given correlation, so it sort of inherently assumes that all pairs are similar an some sense. It would not work well, say, with known block sets of highly ...

2

Quantile regression is considered a robust procedure but lacks the quality of being fully differentiable. There are also regularized regression models like ridge regression, lasso regression and elastic net regression that implicitly consider the covariance of the data like OLS, but additionally reduce volatility in estimates through the introduction of bias....

Top 50 recent answers are included