Suppose you want to calculate the beta of a stock to an index using weekly returns. If the stock is sufficiently volatile, and you use few enough observations, it is possible that the absolute value of your beta estimate will be high even if there is no relationship between the stock and the index (i.e. the real beta is zero). So if you cannot reject the null hypothesis that beta is zero with enough confidence, you should probably assume that beta is zero.
But it is also possible that the actual beta is in fact high. Even though the confidence intervals around your beta estimate are large, your estimate of beta is still the best unbiased estimate you have, even though the variance is large. So then you should probably use the estimate of beta that you have.
Let's say you are using the betas to hedge your concentrated portfolio. How should you manage this dilemma? Should you assume that the stock is uncorrelated with the index and that beta is 0 as there is no significant evidence that it is not 0, or should you use the best unbiased estimate that you have? It seems like a bias-variance tradeoff.
If it depends on a particular use-case, are there examples where either decision would be preferable?