Assuming that the CAPM holds, the total risk of an asset can be partitioned into systematic risk (associated with the market factor) and idiosyncratic risk. Idiosyncratic risk is asset specific. Does that imply that the covariance matrix $\Sigma_t=\text{Var}(\varepsilon_t)$ from a cross-sectional regression $$ r^*_{i,t}=\alpha_i+\beta_i r^*_{m,t}+\varepsilon_{i,t} $$ is a diagonal matrix?

If so, would it make sense to impose this diagonality when testing the CAPM (e.g. via GMM as discussed in Cochrane "Asset Pricing" (2005) Part II), e.g. when testing $H_0\colon \alpha_i=0 \ \forall i$? (I believe the test statistic involves an inverse of $\hat\Sigma$ where the latter "covers" all time periods $t=1,\dots,T$.) I am interested in this, as I hope it could alleviate the problem of inverting a large unrestricted estimated covariance matrix when the amount of time series observations is quite small.