6
Yes. It comes from a core theorem of statics, Stein's Lemma. It shook the foundations of the field of statistics when it came out. It blew up an entire way of viewing mathematical statistics. Although it followed from critical work by Robbins in Bayesian estimation, Stein's work is really what is now remembered.
There are three primary schools of ...
6
The Risk Parity portfolio will be equal weighted if the assets have uniform correlation and equal variance. This would be the case for the shrunk covariance matrix if the shrinkage coefficient used equals unity.
In sklearn, you can check the shrinkage coefficient for the Ledoit-Wolf shrinkage after fitting it from the instance's .shrinkage_ attribute. If the ...
5
here is how to get covariance matrix from correlations:
5
There's more than one way to shrink a covariance matrix. You can think of shrinking a covariance matrix as part of general class of estimators that limit the norms of a matrix. You could alternately think of shrinkage as a form of Bayesian analysis. Given the broad set of techniques one could use, it can be more helpful to think in terms of techniques to ...
4
When using the estimated covariance in the context of mean-variance optimization, then, yes, shrinking the covariance matrix is useful even when you have sufficient data.
A good reference is Golts and Jones, A Sharper Angle on Optimization, who discuss convariance shrinkage among other techniques and give two examples of the usefulness of shrunk covariance ...
3
Let $f(N) = \frac{1}{2} N (N + 1)$ then $f(50) = 1275$. A year has approximately 255 trading days. So you need at least 1275 / 255 = 5 years.
I believe the rule above is used in practice but I think the text is not quite correct (which surprises me, maybe I should have a ☕). If the returns are IID, 51 observations ought to be enough, see the proof in this ...
3
OK, so think of it this way...
Your standard (Markowitz) covariance matrix is a sample observation. That may or not be close to the population sigmas and correlations of your sampled markets. Even if close, the sample-vs-population errors will create asset allocation errors.
The identity matrix here is the "complete strategic ignorance" covariance ...
3
You can think of it in Bayesian terms.
To start with, knowing nothing at all about stocks, you might assume that stock returns are i.i.d with unit variance. This would be your prior. It is very simple and is well behaved because the identity is invertible.
Then you would gather some empirical data on stock returns and measure the actual variances and ...
3
I thought I would answer the question of "what am I using." All shrinkage estimators map to a Bayesian estimator that differs only in the prior distributions. In other words, you get a point estimate that is indistinguishable from a Bayesian estimate except that the calculation rule determines the prior distribution. Stein estimators for the Gaussian are ...
3
PYTHON
I have found this class from the statsmodels library for calculating Garch models. Unfortunately, I have not seen MGARCH class/library.
Below you can see the basic information about the garch models in mentioned class from the statsmodels. Probably you have to implement it by your own in python, so this class might be used as a starting point.
...
3
Broadly speaking, as you probably already know, there are 2 approaches to estimating large covariance matrices:
1) Shrinkage Methods like Ledoit-Wolf that try to reduce the noise in a large matrix (N by N) that has been estimated using the conventional method.
2) Factor Models of Covariance as described in for example Connor Korajczik 2007 that assume that ...
3
The question you asked can be explained by these two lines of the code
e means <- t(returns) %*% ones / T
z <- returns - matrix(rep(t(means), T), ncol=N, byrow=TRUE)
term.1 <- t(z^2) %*% z^2 e
Here returns is TxN which gives you matrix ${y_{nt}}$ where n has i and j elements ; means is TxN of matrix ${z_i}$ , same mean for each asset for the ...
2
As an addition to the already rich answers, I would suggest you to read the following paper by Marcos L. De Prado on the computation of Forward-Looking Correlation Matrices.
Estimation of Theory-Implied Correlation Matrices
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3484152
2
The Ledoit-Wolf estimate cited by @develarist can be quite good, but as you say you already knew about "shrinking". It takes the population of correlations observed as an effective Bayesian prior for any given correlation, so it sort of inherently assumes that all pairs are similar an some sense. It would not work well, say, with known block sets of highly ...
2
Quantile regression is considered a robust procedure but lacks the quality of being fully differentiable. There are also regularized regression models like ridge regression, lasso regression and elastic net regression that implicitly consider the covariance of the data like OLS, but additionally reduce volatility in estimates through the introduction of bias....
2
The relationship between covariance, standard deviation and correlation is:
$$ corr(x,y) = \frac{cov(x,y)}{\sigma_x \sigma_y}$$
So to construct your matrix you will have the variances in the diagonal:
$$ cov(x,x) = corr(x,x) \times \sigma_x \times \sigma_x = 1 \times \sigma_x^2 = \sigma_x^2 $$
And for the covariances:
$$ cov(x,y) = corr(x,y) \times \...
2
Generally it is better to shrink the covariance matrix—since the variances of your data probably vary a lot, and the correlation matrix treats them all as essentially equal variance, you throw out the baby with the bath water by pausing to the correlation matrix. In effect, when you shrink the correlation matrix, you correct a lot of stuff that is not ...
1
The covariance matrix of $N$ stocks (or whatever) consists of $N(N+1)/2$ distinct elements, so, to statistically measure these elements reasonably well, your number of independent observations $ND$ ($D$ being the number of days) should be well over $O(N^2)$, or $D\gg N$. This requirement is more stringent than the covariance $C_{ij}=\sum_dR_{di}R_{dj}$ ...
1
Let your linear system be defined by the latent multivariate state variable $x_t$, progressing in an AR(1) fashion, and let your observation at time step $t$ be linear in the latent state:
$$
\begin{align}
x_{t+1}&=Ax_t+u_t\\
y_{t}&=Hx_t+v_t\\
\end{align}
$$
with $u_t\sim N(0,Q)$ and $v_t\sim N(0,R)$ time-and-state-independent process noise.
Let $K_t$...
1
Hi: Exponential smoothing weights observations by taking a weighted combination of the old estimate and the new. So, if you denote your original matrix ( or current covariance matrix ) as $R_t$ and your new one as $R^{*}_t$, then exponential smoothing does
$R_{t+1} = \lambda R_{t} + (1- \lambda) R^{*}_t $.
But there are two issues with doing this update.
...
1
Yes all you have to do is estimate the Black Litterman covariance matrix that includes investor views using a shrinkage estimator. Covariance shrinkage like Ledoit Wolf is an old technique, however, that has been outperformed by the denoised or detoned covariance matrix estimated by random matrix theory, as well as the nested clustered optimization (NCO) ...
1
Usually, when one talks about exponential smoothing, they talk about it's halflife.
So, for example, suppose we exponentially smooth some quantity ( argument carries over to covariance matrix but I'd rather just rather consider the scalar quantity case ) and call the exponentially smoothed estimate $\hat{smth_t}.$
So, this means that we have:
$\hat{...
1
I have actually considered the problem that you are working on, though configured somewhat differently.
There isn't going to be a universal answer to your question. See, in particular,
Holland, Paul W. Covariance Stabilizing Transformations. Ann. Statist. 1 (1973), no. 1, 84--92.
Nonetheless, there are answers, some already mentioned. I would argue ...
1
This is not a complete answer, more a different perspective to the answers already given. If you have some a-priori knowledge about the covariance structure and about the factors influencing it, you should try to reflect this in your statistical model. Three ideas:
Divide your sample into subpopulations with identical factor values and estimate separately. ...
1
this answer is on hold
first it used the fact that your function $y$ is symmetric around 0 (proof) can be found here, so i don't need to type everything.
then just expanding the summation $$lim_{n \rightarrow +\infty} n^{-1} \sum_{|h| < n} \left(|\gamma(h)| \right) = lim_{n \rightarrow +\infty} n^{-1} * \frac{(y(-n)+y(n))*2n}{2} $$ because h is from -n to ...
1
I recently met the same problem and found a way to achieve it using R in Python.
from rpy2.robjects import pandas2ri
import rpy2.robjects as objects
import numpy as np
# pd_rets - a pandas dataframe of daily returns, where the column names are the tickers of stocks and index is the trading days.
# compute DCC-Garch in R using rmgarch ...
1
Slight correction: the package in R is called rmgarch, not mgarch. It works well with rugarch, which provides a variety of univariate GARCH models. Both packages allow for parallelized computation on local cluster and return a nice and full set of fitted parameters, model specs, etc. I provided some additional links in this post.
1
The implementation is explained in more detail in the Horse - Race of DeMiguel: see here
1
Say that you did the calculations in the classic regression way. If you stick the returns of your 4 asset returns in a $(T\times 4)$ matrix $Y$, and your 3 factor returns in a $(T\times 3)$ matrix $X$, then your betas would solve the multiple regressions, collected in a $(3\times 4)$ matrix
$$Y = X\cdot \beta + \epsilon$$
You could also add a column of ones ...
1
With this solution you have to split your covariance matrix somewhat, but it should give you a vector with betas based on you conditional covariances.
Example with two indexes, $x1$ and $x2$, and one asset $y$.
$$[\sigma_{y,x1}, \sigma_{y, x2}]\begin{bmatrix} \sigma_{x1}^2 & \sigma_{x1,x2} \\ \sigma_{x1,x2} & \sigma_{x2}^2 \end{bmatrix}^{-1}$$
Only top voted, non community-wiki answers of a minimum length are eligible