PCA risk modelling

Question

Been doing loads of reading about PCA, FA and SVD but still fail to understand the fundamentals of how PCA links with factor analysis in the context of risk modelling. Here is where I'm stuck:

Given a TxN matrix, each row is a time period, column is an asset. This is returns in the basis of assets. We center each column and find the eigenvectors of the cov matrix. Then regress out the eigenvectors. The regression coefficients are thought of as factor returns and an eigenvector (if endowed with variance via sqrt(eigenvalue)) give exposures of each asset to the factor (portfolio) associated w that eigenvector.

In factor modelling, we say each factor is a portfolio (weighted combination of assets), its returns are the returns of the assets weighted by the weights which define the portfolio.

Mathematically, the eigenvectors are a basis. What variance is it that they maximise here? Is it that the eigenvector with the highest eigenvalue includes exposures of assets to a portfolio with the highest possible risk - dispersion of returns over time ?

Next, why do I need regression? Can't I just project my original variables (assets) onto the subspace spanned by SOME of the factors, and keep only that (or remove it to get residual returns).

Scale: I've seen many answers about "endowing the eigenvectors with scale to get loadings" by multiplying them by the sqrt(eigenvector), so that their squared norm gives the variance. What role does this play in the factor model exactly? Finally, I've seen legacy code at work where they standardise X first (divide each column by std) and at the end rescale it back up. I've also seen examples where they DON'T scale it back up in the end, what is the purpose of that and the difference between the two?

Answer 1

A few comments:

1. Suppose you have eigenvectors of a covariance matrix. One way you can think of the first eigenvector is the vector of length $1$ where, if we held a portfolio whose weights were given by that vector, the variance would be maximized (ie no other vector of length $1$ would give a greater variance.) The second eigenvector is the unit-length vector orthogonal to the first eigenvector that gives the greatest variance. And so on.

2. Projection is a slightly different operation from regression. Here's a link that I think has a nice visual. https://blogs.sas.com/content/iml/2020/02/05/visualize-residuals-for-various-regression-methods.html The first graph in the three is classic "regression" while the third is classic "projection". (The middle one I guess is a different kind of regression but not one that I see used!) I think the distinction is important if you are trying to do statistical inference, in practice it depends on the use but you might find they give results that are different but not completely disagreeing.

3. I'm not the best person to answer this, I will say for now that scaling in the covariance matrix will impact the vectors that come out, meaning you'll end up with a different basis. Different scalings can have different uses.