4
$\begingroup$

I have 1 dependent variable and 3 independent variables.

I run multiple regression, and find that the p value for one of the independent variables is higher than 0.05 (95% is my confidence level).

I take that variable out and run it again. Both remaining independent variables have $p$-value less than 0.05 so I conclude I have my model.

Am I correct in thinking that initially, my null hypothesis is

$$H_0= β_1=β_2 = \dots =β_{k-1} = 0$$

and that the alternative hypothesis is

$$H_1=\textrm{At least one } β \neq 0 \textrm{ whilst } p<0.05$$

And that after the first regression, I do not reject, as one variable does not meet my confidence level needs...

So I run it again, and then reject the null as all $p$-values are significant?

Is what I have written accurate?

Edit: Thanks to Bob Jansen for improving this aesthetics of this post.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
3
$\begingroup$

The hypothesis $H_0: β_1=β_2=\dots =β_{k−1}=0$ is normally tested by the $F$-test for the regression.

You are carrying out 3 independent tests of your coefficients (Do you also have a constant in the regression or is the constant one of your three variables?) If you do three independent tests at a 5% level you have a probability of over 14% of finding one of the coefficients significant at the 5% level even if all coefficients are truly zero (the null hypothesis). This is often ignored but be careful. Even so, If the coefficient is close to significant I would think about the underlying theory before coming to a decision.

If you add dummies you will have a beta for each dummy

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your response. I don't have a constant, all of my p-values are very significant (the least is a dummy variable at 0.039). What would my null hypothesis be? My knowledge is that I'm seeking p-values because that'd give me my model. I don't understand the technicalities of it and want to learn it :) $\endgroup$
    – Harry
    Jan 7, 2015 at 22:36
  • $\begingroup$ I think you meant to say 14% of committing a type one error (probability of 0.14 of finding at least one of the coefficient significant when there true value is actually the null hypothesis value) $\endgroup$
    – Kamster
    Jan 8, 2015 at 0:36
  • $\begingroup$ @Kamster Thanks. You are correct and I have amended my answer. $\endgroup$
    – user1483
    Jan 21, 2015 at 21:26
2
$\begingroup$

These are independent variables so the hypothesis applies to each parameter independently.

Improve this answer
$\endgroup$
8
  • $\begingroup$ +1: Yes, you are right - but the rest of it should be fine $\endgroup$
    – vonjd
    Jan 2, 2015 at 21:18
  • $\begingroup$ sorry, could you clarify? How do I change the equation so it applies to each parameter independently? And also, what is the effect of adding 3 dummy variables. Is it simply 2 more betas? Or do they require their own symbol $\endgroup$
    – Harry
    Jan 4, 2015 at 0:32
  • $\begingroup$ It just means that you have an H_0 and an H_1 for every parameter. $\endgroup$
    – vonjd
    Jan 4, 2015 at 11:33
  • $\begingroup$ Ok I see. Do you know the procedure for dummy variables? Are they just additional beta? Or is it more accurate to refer to them as delta? $\endgroup$
    – Harry
    Jan 4, 2015 at 11:43
  • $\begingroup$ Maybe I have this wrong but isn't it true if you remain your individual significance levels at 0.05 that the probability of type one error (ie the probability that reject null hypothesis when it is actually true; significance level) will be greater than or equal 0.14 $\endgroup$
    – Kamster
    Jan 8, 2015 at 0:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.