Or should Fama-French only be applied to portfolios?
In theory, the Fama-French model --- a linear factor asset pricing model --- applies to ALL assets (in particular, a single stock is a portfolio with one stock as its holding).
In theory, each cross sectional equilibrium model applies to every single financial asset, therefore to single stocks, bonds, options, other derivatives etc… Fama-French’s model doesn’t work when tested on financial assets other than 25 size-value portfolios, as shown here or here. Therefore it’s very likely it won’t hold on single stocks, too.
One side note: the reason why cross-sectional models are tested on portfolios is to average out all the noise involved in estimating the betas. But when you construct a portfolio you need to be sure there is a valid reason to believe that it will lead to more stable betas. Check this lecture by Michael Brandt in which he discusses about this issue.
If the Fama-French model were the correct asset pricing model, it could be applied to any return, whether it is the return of a portfolio or the return of an individual stock.
That said, there are some issues if you want to apply factor models at the firm level.
A big problem: imprecise measurement
The problem is that betas of individual stocks as estimated from a standard time-series regression tend to be measured extremely poorly.
- There's so much volatility and noise at the individual company level, your standard errors are going to be quite large.
- Betas almost certainly change over time. Apple in the mid 1990s is a different company than Apple today. Using a longer time series to estimate betas at the company level is problematic too.
A motivation for forming portfolios on characteristics and estimating the betas of the portfolios is that hopefully the portfolios have more stable factor loadings over time and that much of the noise is diversified away. With highly diversified portfolios, the FF3, FF5, etc... factor model regressions actually can give a quite high $R^2$.
Some ideas of what to do?
It really depends on how you're using the betas, what your question is. If you're interested in ONE particular company, the brutal truth may be that using a market beta of 1 may be about as predictive going forward as any market beta you estimate. (Mechanically, the value weight average market beta must equal 1.)
- Go Bayesian: put a prior on your beta estimates and do a Bayesian linear regression.
- Extreme prior: ignore time series regression and use market beta of 1.
- I've seen corporate finance Prof. Aswath Adamodar at NYU advocate using betas estimated from firms in the same industry?
- There are numerous papers that show firm characteristics are highly related to factor loadings.
On the other hand, using betas estimated from time series regressions on individual companies could be good enough. It really depends on what you're doing.