Missing factor in the factor model

I am developing a factor model to predict monthly returns. One of the factors alone accounts for an R squared of 0.3 to 0.4 for many single periods that has surprised me.

However, for some periods the direction of the factor reverse completely and if I run a single regression for all the periods this factor alone accounts for an R squared of 0.1 which means that the factor is robust but I am missing another factor, correct me if there is another explanation.

Does anyone have any opinion on how to isolate the explanatory power of this factor in the panels in order to eliminate the effect of missing factor or any analytical way to flag the other factor?

• There is not enough information for me to answer it but - your statement One of the factors alone accounts for an R squared of 0.3 to 0.4 for many single periods that has surprised me. Statistically it is not surprising and I need to know more about the factor and what you are modelling and your assumptions are to pin point if there is anything wrong with it If you think it is important factor than have it by all means. See the outliers and try to analyse them.
– ash
Nov 16 '12 at 9:38

The best option is to identify the other missing factors and include them in your analysis. Depending on your data and assumptions, PCA is a good place to start.

Your data also shows signs of a time-varying correlation with your factor. Hence, it $may$ also be appropriate to allow for time-varying regression coefficients or some other technique to account for this feature of the data.

• Right John. I introduced N dummy variables for each period to isolate the time-factor, R squared increased from 0.1 to 0.22 which confirms your hypothesis. I also was thinking to take a shorter period of time. Right now I am running the model for 43 months ending this October, but this force the Beta to be stationary over almost 4 years, which might not be necessary in practical usage of the model. Do you have any comments. about the missing factor, I have no idea how to identify that factor, I should try PCA, but interpreting that factor is another pain in bott. Nov 17 '12 at 0:10
• I'd have to know more about what the data is like.
– John
Nov 17 '12 at 0:13
• send me your contact if you would like to take a look at the data. They are monthly returns of S&P100. Nov 18 '12 at 0:13
• Not necessary. The returns on the index should explain a significant amount of the variation, but PCA can also help.
– John
Nov 18 '12 at 1:39
• This is a predictive model, I am locked to the factors are available in the preceding period. However, I added index returns it could not help the model as I expected. Nov 18 '12 at 2:35