7
$\begingroup$

Let's say I have the following regression setup, which I am using for portfolio return attribution:

$R = 1*\beta(1) + A*\beta(2) + B*\beta(3) + C*\beta(4) + \epsilon $

where A is dummy matrix of country , B is a dummy matrix of Industries, C is a matrix of factor exposures, 1 is a vector of ones

As you can see in this setup, there is multicollinearity between A and B. The rank of (AB) < rank of (A) + rank of (B). In Matlab code:

rank([dummyvar(ceil(abs(rand(20,1))*5)'),dummyvar(ceil(abs(rand(20,1))*4)')])

How can I go about computing all these betas without dropping one of the Columns from A or B matrix, as is typically done to address this issue. I know there is a trick to solve this, but I can't remember how it's done.

$\endgroup$
  • 1
    $\begingroup$ Why is it clear that there is multicolinearity? $\endgroup$ – phdstudent Feb 21 '16 at 17:27
  • $\begingroup$ See the edit above. $\endgroup$ – silencer Feb 22 '16 at 16:10
2
$\begingroup$

To address the issue of multicollinearity you can basically do the following (source, the author wrote his PhD thesis about multicollinearity diagnostics):

  1. Getting more data
  2. Dropping one variable
  3. Combining the variables (e.g. with partial least squares regression or principal component regression) and
  4. Performing ridge regression, which gives biased results but reduces the variance on the estimates.

I don't know whether 1. is feasible but it is always a good idea (one of the learnings of data science is that getting more and better data is preferable to more sophisticated algorithms).

Concerning 2. you said that you don't want that.

3. doesn't seem practical either since the new variable won't be interpretable any more.

So 4. seems to be the best candidate. Mathematically this is also known as Tikhonov regularization. In matlab a good starting point seems to be this one:
http://www.mathworks.com/help/stats/ridge-regression.html

The command is ridge(), see also here:
http://www.mathworks.com/help/stats/ridge.html

$\endgroup$
1
+100
$\begingroup$

You can use a constrained regression. This will address exactly your problem. It is afaik the industry standard way of addressing this problem.

You will want to set up a constraint for each of the group of dummy variables you have such that the sum of the coefficients for that group is 0. In this case for A and B. You can leave C without constraint (you should actually !)

If the co-linearity is coming from C (factors). Then it's another issue and advices in other answers are then very useful.

You need to do this as long as you have more than 2 groups, or one group and the intersect. It's pretty standard as a risk modelling technique. See for example how MSCI deals with it in their US model msci.com/documents/10199/ed6e42a3-c1fa-4430-89ba-efd5a5b52558 (middle of page 1)

On a side note, you should also consider using a robust regression technique.

If you are using Matlab (it seems from your snippet) you can read about constrained and robust regression in this excellent tutorial: http://www.mathworks.com/matlabcentral/fileexchange/8553-optimization-tips-and-tricks

And use the following function: http://www.mathworks.com/matlabcentral/fileexchange/13835-lse Although Matlab also has some regression functions but are more limited IMO.

$\endgroup$
  • $\begingroup$ Although constrained regression can be useful in some circumstances I fail to see how this can address this specific problem of multicollinearity. Can you give a source? (I didn't find any references concerning this point in the documents you cited). And could you please also give a source for your claim that it is the "industry standard for addressing this problem"? Thank you $\endgroup$ – vonjd Feb 26 '16 at 17:23
  • $\begingroup$ It addresses the problem if the multicolinearity comes solely from A and B (ie the dummy variables). By adding the constraint you are removing the extra degree of freedom. Right ? $\endgroup$ – NegativeJo Feb 26 '16 at 17:50
  • $\begingroup$ As for industry standard other than 'trust me I am an expert' :) I dont have any documents that I can freely share... I'll try to google one though that links to this $\endgroup$ – NegativeJo Feb 26 '16 at 17:51
  • $\begingroup$ Well, removing the extra degree of freedom by adding constraints to certain variables seems a little forced (and certainly not "standard"). Ridge regression is a far better alternative to reduce the variance in my opinion. I am curious if you find something on the net. $\endgroup$ – vonjd Feb 26 '16 at 18:03
  • $\begingroup$ But wouldnt the ridge regression make the less relevant dummy variable for each group coefficient go towards 0 ? What if you wanted to force each of the dummies to have a coefficient (for a more interpretable country by country attribution for example)? Interesting point though. $\endgroup$ – NegativeJo Feb 26 '16 at 18:16
0
$\begingroup$

You can use Principal Component analysis which will divide the variable into non-correlated factors and the use the factor summary scores as an input in the model instead of variable scores.

$\endgroup$
  • $\begingroup$ The problem with PCA is that in this case the new variables won't be interpretable any more (see also point 3 in my answer). $\endgroup$ – vonjd Feb 26 '16 at 9:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.