Let's say we are working with the standard Fama-French 3 factor model and we want to add a low volatility factor. Is it alright to add a low volatility risk premium in a model such as the CAPM or FF3. Are we violating the CAPM assumption that high beta stocks are more risky and have a higher expected return? I am wondering if the model is still valid and if not, is there a solution?
The CAPM claims that only systematic risk matters (i.e. covariation with the market) to determine an asset's expected return. So the fact that low volatility stocks have returns that are not explainable by market beta is an empirical contradiction of the CAPM to start with. The CAPM is too rigid and performs poorly in explaining the cross section of equity returns. Other examples include value, size, momentum, profitability, asset growth, illiquidity, higher moments, seasonal patterns, low beta, payout, lotteriness ... It is an endless list. Lately since Fama and French (1992) we know that the CAPM is dead.
Because of the above findings, Fama and French (1993) proposed a new factor model, adding value and size as additional risk sources which proxy the impact of (unknown) state variables on expected returns. Thus, the CAPM's empirical failure is the only reason why the FF3 model exists.
The Fama and French three-factor model performs much better than the CAPM (market model) but is not perfect either. An important problem is momentum (which lead to the FFC model from Carhart (1997)). Fama and French (2015) and Hou et al. (2015) propose five- and four-factor models including some sort of profitability and investment as additional risk factors.
Thus, it is absolutely okay to add further factors to the CAPM and the FF3 model. Fama and French did it themselves. It simply means that you believe that the space of risk drivers is not just one- or three- dimensional but includes further risk sources.
The only question is whether it is sensible to include further factors. You could also include a factor based on the first name of the CEO etc. That factor would probably be useless. So you should have a good reason to add further factors.
The above may sound harsh, the CAPM is a huge milestone and breakthrough in finance and economics. It was rightfully awarded with the Nobel Prize. It's just not quite perfect.