Podcast #128: We chat with Kent C Dodds about why he loves React and discuss what life was like in the dark days before Git. Listen now.
18

Quant Guy's list is really impressive! However, I am not sure they will readily solve your specific problem? I think there is one missing piece. Please note that imputing missing data is a very broad topic. There are many recipes to impute missings but that's for their specific 'assumptions' and purposes. They do not necessarily intend to well address your ...


18

There is some research that directly bears upon the issue of estimating covariance in the presence of unequal return histories and regime change. Wharton professor Robert Stambaugh (of liquidity premia fame) wrote a paper in '97 called "Analyzing Investments Whose Histories Differ in Length". Prior to the paper, most academics and practitioners would use "...


12

Yes, you need Cholesky factorization. You can find the general idea here: http://www.goddardconsulting.ca/option-pricing-monte-carlo-basket.html Plus the implementation in MATLAB here: http://www.goddardconsulting.ca/matlab-monte-carlo-assetpaths-corr.html The code in general should be easily translatable. The only difficulty is the Cholesky factorization ...


9

There are two functions for estimating variance matrices with missing values (and aimed at finance, by the way) in the R package BurStFin. Available via: install.packages('BurStFin', repos="http://www.burns-stat.com/R") but not yet for 2.14.x. You can of course get a correlation matrix from the variance matrix. One function estimates a statistical ...


9

For years, I performed this brute-force search daily on my universe of tradable stocks and futures. It is a waste of time. If your computer discovers that hog futures and MSFT are cointegrated, for example, do you really care? I would never trade that pair. There is no economic connection between hogs and Microsoft, so I must assume that the reported, small ...


9

If $X \sim N(\mu, V)$ is multivariate gaussian, you can write $X = \mu + C Y$ where $ Y \sim N(0,1) $ is a standard Gaussian and $C$ is the lower-triangular Choleski matrix of $V$. You can then express $ v = \sum_{i=1}^n (X_i - S/n)^2 $, where $ S = \sum_{i=1}^n X_i $, in terms of $Y$ and $C$. (I do not reproduce the computations: they are straightforward.) ...


9

This is correct: "The general idea of cleansing a correlation matrix via random matrix theory is to compare its eigenvalues to that of a random one to see which parts of it are beyond normal randomness." This is not correct: "These are then filtered out and one is left with the non-random parts." The term "filtering", although used extensively in the ...


8

Theoretically, the answer to the question is yes, a correlation matrix for potential pairs trades can be computed in $O\left((n^2t)^{(\omega+\epsilon)/3}\right)$ time, for any $\epsilon > 0$, where $\omega < 2.38$ is the so-called exponent of matrix multiplication. However, these algorithms have a reputation for having a very large constant factor ...


4

For the stationary multivariate normal case, the expected returns vector does not matter. This is because the cross-sectional mean is subtracted out before calculating the standard deviation. The cross-sectional mean can be more conveniently thought of as like the return on an equally weighted portfolio. Similarly, I would argue that the expected cross-...


4

He is forced to use some tricks because Excel can only take average of a rectangular area, but he wants the avg of upper non-diagonal elements of the matrix only. So he subtracts $\frac{1}{n}$ (the average of the 1's on the diagonal), then scales the result by $\frac{n^2}{n(n+1)/2}$ which is the number of total elements divided by on-or-below-diagonal ...


4

For ex-ante tracking error, you need a forecast covariance matrix $C$. Then the quantity you require is $\sqrt{w^{T}Cw}$, where $w$ is a vector of excess weights relative to the benchmark. You can construct a forecast covariance matrix from realized covariances if you think historical relationships will persist, or you use other methods, for example factor ...


3

It's probably easiest to think about it in terms of a covariance matrix and then convert it to a correlation matrix after. If instead of the first matrix you have some covariance matrix of the assets $\Sigma$, then you could get the portfolio variance, for one portfolio, as $w' \Sigma w $, where you could have $w\equiv\left(w_{1},w_{2},w_{3}\right)'$. ...


3

How about an O(N log(n)) solution ? To be a viable trading strategy, you often expect them variances to be similar, so just calculate ordinary volatility and put it in an ordered array. Of course that's going to be period dependent, so pick a few arbitrary periods and see which instruments end up being together. Then you get clusters of vastly smaller ...


3

I think Cholesky on correlation matrix is better because it makes code apply more generally in case we don't have full rank. For example, suppose we want to simulate three correlated normals with covariance matrix [[a^2,0,0], [0,b^2,0], [0,0,c^2]] i.e. variables are uncorrelated and have vols a, b, and c. Because this is positive definite, we can do ...


3

You can use the either, as both necessarily are symmetric positive definite; covariance is a personal preference. It's really just a matter of scaling, as $\mathcal{N}(0,\Sigma)$ is distributionally $\sqrt{\Sigma} \mathcal{N}(0,1) $. Correlation would require additional scaling (i.e. multiplication of every $\mathcal{N}(0,\rho)$ element by its respective ...


3

How about adapting Ledoit-Wolf shrinkage to average correlation? You calculate the ratio of average correlations in two regimes to get a sense of the magnitude of regime shift. Use this ratio to adjust the correlations of short-lived stock with others. The method is simple and result will definitely make sense to you.


3

I do not understand why t is zero in B(t,t) and 1,2,3,4 in B(i,t). And why did you interprete the value B(i,t) as the inverse of C, but not B(t,t)? I think B(i,t) is the first column of I. An inverse is defined by Matrix^-1. But, B(*) is a value. So, you should divide by B(t,t). Here, you multiplicate it.


2

Let $t$ be the number of days (time periods), and let $p$ be the number of assets. You have $t=1000$ and $p=10000$. For any given dataset, it is assumed that the sample covariance matrix $\mathbf{C}$ accurately represents the population covariance matrix $\boldsymbol{\Sigma}$, however, as $p \rightarrow t$ or if $p > t$ (as in your case), the ...


2

I think that your problem can be solves by using another estimator for your covariance matrix. A so called shrinkage estimator leads to covariance matrix that is non-singular. Then a Cholesky decomposition should work (maybe there is even a short-cut in the shrinkage world, I will check alter on). The R package corpcor contains functions to perform ...


2

I am not an expert in this field, but it would be best to consider a full multivariate GARCH model. This paper by Engle and Sheppard should be a good start. I think the constant correlation matrix approach is covered to a certain extent too. I hope this helps.


2

I had to answer because of your name, and becaue I deal with portfolio optimization often. In my world of equities correlation does matter a lot. If one follows the thoughts e.g. here then it matters most. I deal with minimum-variance construction (no expected return, of course some constraints on the weights) and there I often see positions that come ...


2

See "Some hypothesis tests for the covariance matrix when the dimension is large compared to the sample size" by Ledoit and Wolf. http://projecteuclid.org/DPubS?service=UI&version=1.0&verb=Display&handle=euclid.aos/1031689018


2

Your issue demonstrated in R with interesting solution Equity Risk Model Using PCA. Another useful link in Matlab by Nick Higham himself NCM implementation by Nick Higham written for Matlab. Another good discussion on shrinkage and other aspects of Correlation Adjustment


2

There is a wide knowledge on correlation estimation, see other questions and answers: principal component analysis (PCA) - Equity Risk Model Using PCA random matrix theory (RMT) - Cleansing covariance matrices via Random matrix theory or Random matrix theory (RMT) in finance shrinkage - Portfolio Optimization : Shrinkage of Covariance Matrix when data is ...


2

You have the risk factor $F$ and the asset that it is correlated to $r_m$. You can calculate the variances of each of these, say $\sigma^2_F$ and $\sigma^2_m$. If you do not care about the distribution but just work with variances and correlations then can look at an OLS setting: $$ F = \beta r_m + \epsilon $$ with $\beta = \rho \frac{\sigma_F}{\sigma_m}$ ...


2

If you look at changes of the points on the yield curve, then you probably find something stationary - right? Applying PCA on the covariance of these changes makes sense. E.g. you will find out that on PC describes a parallel shift (a change in the yield curve). Look at this question too: What do eigenvalues/eigenvectors of the yield/forward rates ...


2

There is a vast literature on modelling time-series with periodcities. Rob Hyndman is one of the leading reseaerchers in this area. He has published the R package forecast and a free online text book on this subject (with another package and R code in the book). Your task is covered starting here.


2

I don't know how common this is, but I've seen it done. Many risk model vendors (Northfield, Axioma) allow the blending of different risk models with different periodicity (e.g. a shorter horizon risk model blended with a longer horizon risk model). Here's a Northfield deck about this: https://www.northinfo.com/documents/779.pdf


1

If I understand correctly what you are after is the marginal volatility contribution of a single asset to the portfolio. This is given by $$ \sigma(X_j;X) = \sigma(X_j)\ \rho(X_j, X) $$ See here for details.


1

You will need the covariance matrix to calculate this. Say you have a collection of $n$ assets. The value of asset $i$ is represented by the random variable $X_i$ and the corresponding portfolio weight is are $w_i$, and $v_i$ for the two portfolios. The correlation between the two portfolios is: $$ \frac{\sigma(w^TX,v^TX)}{\sqrt{(w^T\Sigma w)(v^T\Sigma v)...


Only top voted, non community-wiki answers of a minimum length are eligible