Suppose I have a factor model
$$ \rho_i = \sum_J \beta_{iJ} \rho_J + \epsilon_i $$
where $\rho_i$ is the excess return of asset i over the risk free rate and $\rho_J$ is the excess return of the factor J portfolio. My J's are fixed and i's are all the possible stocks.
How would I go about verifying the model? Concretely, it is a linear regression problem and I can find out the $\beta_{iJ}$ and their statistical significance. If it were just one stock I were regressing against the statistical significance would tell me if the model is good. However, for multiple stocks what is the criteria? I am sure this can be answered using stats but I want to know if this is a well known result or should I sit down and compute it.
The motivation for this is that CAPM and Fama-French were based on some intuition for the factors (and that is how it should be) but in the age of cheap computation and AI one might find a factor and it will be useful to verify it across a large universe of stocks.