Nice question! I don’t have a precise answer to it but I will try somehow to give you my thoughts.
I think it depends a lot whether you have in mind an APT or an ICAPM as explained in this article by Eugene Fama. The APT is really agnostic regarding the risk premia and the factors, and basically the only prediction is that alphas are going to vanish thanks to arbitrage between a diversified portfolio and a risk free bond: what is crucial is that there is no theory behind the origins of the factors.
On the other hand, if you have in mind an ICAPM extra factors should hedge investors against some state variable governing the stochastic investment opportunity set they are going to face in the future. So, in this case, we should somehow justify why the factor is able to hedge you against some risk.
Fama and French try to give some economic argument behind their factors but they are not convincing, that’s why you should probably see their model as just an APT. The same is true for Carhart’s model.
Regarding the difference between factor and anomaly, there is clearly nothing that prevents you to put the extra return of an “anomaly” on the right hand side and check whether it is priced if you have in mind an APT. On the other hand, to do the same with an ICAPM you should show that your anomaly is somehow linked to predictability, that it hedges some systemic risk or, possibly, invoke the factor mimicking theorem and convince people your factor is nothing more than the projection of the true SDF on the payoff space.
All in all, there has been some movement recently regarding these topics. In my opinion, the most convincing ICAPM that has been recently proposed is the one by Adrian, Etula and Muir, published in the last issue of the Journal of Finance, where the pricing factor is the innovation in broker-dealers leverage, mimicking for the risk of fire sales due to the prociclicality of Broker-Dealers’ leverage.