Consider a scenario in which Y_t represents the % change in price and we want to use X_t to predict Y_t. We assume that X_t is information we get before Y_t is revealed.
Suppose that in reality Y_t = 0.01*X_t + e_t (beta = 0.01), where e_t ~ N(0,1). Moreover, X_t ~ N(0,1). Suppose we have 1000 points of historical data to test on. This generates the distribution of beta when simulating the estimation 1000 times:
A few questions:
1) Is this approach a feasible money making strategy in finance? First, the assumptions I have made are already really optimistic (Y_t,X_t are iid over 1000 samples and the true relationship is linear). However, because of the noise the variance of the estimated parameter is very high, and 40% of the parameters are the wrong sign. Adding more rhs variables would presumably just make this worse.
Even presuming we got beta = 0.01, the signal is so small that we will have a high error rate of falsely rejecting our signal when the strategy does poorly.
2) What are possible solutions? If we had enough data (HFT?) we could precisely estimate small signals. What if we are looking at lower frequency time series (daily data?). Is another solution to look for a Z_t that has a much higher beta? However, given that markets are relatively efficient, why would a large beta continue to exist? Do we need to add more low-signal variables into the RHS of the regression (this would just increase the variance though)?
Another approach to get a higher "beta" seems to be stat arb? If Y_t and X_t are contemporaneous stock prices of cointegrated price series, we could expect beta to be large enough to estimate more precisely (relative to its magnitude). However, I have read (although I clearly don't have first hand experience) that stat arb profits have been declining since everybody knows the idea nowadays.