It is my understanding that for any given stock, the sample mean of historical returns is not a good proxy for the stock's expected return. In fact, the return on a stock needs to be estimated via some sort of factor model, be it the CAPM, FF3, FF5, etc. Basically, we would run a time-series regression of the stock's returns against the factors, extract the betas, and then multiply the betas by the expected return on the factors and that would give us the expected return on the stock.
But where do we get the expected return on the factors from? In all the problems I have done in school they were always given, but now I am trying to actually do this in practice in order to optimize my portfolio and I am a bit lost. I thought about using the sample mean, but then wouldn't we have the same issue as for the individual stocks (past returns are not an indicator of future returns because returns are not IID)?
Alternatively, could we take the average of the fitted values from the factor model and use that as our proxy for the expected return on the stock?
Also, what about the standard deviation and the correlation of the stocks within in a portfolio? Do we use the sample standard deviation or do we estimate it by applying the variance operator to the factor model regression (Var[R_i] = beta^2 * Var[R_mkt] + ... + Var[epsilon])?
Any help would be much appreciated. It would also be great if you could point me to papers/articles/books that tackle these types of issues.