Given an $n\times 1$ vector of asset returns $r$, and a $n\times k$ matrix of factor loadings $X$, we can express the asset returns in terms of as-yet-unknown factors $f$ using

$$r = Xf + \epsilon$$

and then estimate the factor returns with a cross-sectional regression

$$\hat{f} = (X^TX)^{-1} X^Tr$$

which is equivalent to minimizing the cross-sectional stock-specific return $\|{\epsilon} \|^2$. However, this relies on the loading matrix having linearly independent columns, so that $X^TX$ is invertible.

There are sensible reasons for choosing $X$ to not have linearly independent columns. For example, with 4 stocks from two distinct industries, we may want to consider an overall market factor, and two industry factors -

$$X = \left[\matrix{1 & 1 & 0 \\ 1 & 1 & 0 \\ 1 & 0 & 1 \\ 1 & 0 & 1}\right]$$

Whilst any one of these columns can be expressed in terms of the others, it seems undesirable to remove the market factor, and unsymmetric to remove either of the industry factors.

Taking a cue from ridge regression we could consider minimizing a combination of the idiosyncratic return variance, and the factor return variance -

$$L(\epsilon,f) = \|\epsilon\|^2 + \lambda \|f\|^2$$

$$\hat{f} = ( X^T X + \lambda I )^{-1} X^T r$$
where the inverse is defined for all $\lambda > 0$. My questions are -
2. What is the effect of $\lambda$ on the estimated factor returns? Clearly, increasing $\lambda$ reduces the explained idiosyncratic return, but can we say more than this?
3. Is there an obvious "best" choice for $\lambda$ or does it need to be chosen in an ad-hoc manner?