My Null for the T-test is

h0: -tcritical < Tstat < +tcritical

I require confidence level of 95%.

If my empirical result satisfies the null, but not my p-value requirements,

does this mean that the test is inconclusive? Or that the null is not rejected/rejected?

  • $\begingroup$ This better fits to stats.stackexchange ... mabye much more people will answer there. $\endgroup$ – Ric Jan 5 '15 at 14:12
  • $\begingroup$ Please don't cross post I could have migrated it. I agree it fits better there and I'll close it here. $\endgroup$ – Bob Jansen Jan 5 '15 at 15:31
  • $\begingroup$ This question appears to be off-topic because it is about alreadsy cross posted to stats $\endgroup$ – Bob Jansen Jan 5 '15 at 15:31

That means the null is not rejected and therefore test is inconclusive. With this type of testing you can only try to reject the null. Your non-rejection could have been due to lack of data, therefore you cannot conclude anything from it.

If you want to somehow support your null, compute the confidence interval for your parameters and show that its sufficiently narrow around your null. This will also give you a good overview about the quality of your data.

For example: Suppose you have null that the mean of your distribution is 0 (and the real distribution variance is for example ~1), and you compute the 95% confidence interval for mean as (-0.2,0.3). This means that with 95% probability the confidence interval covers the real mean, which sort of means, that the real mean is not far from the 0 in the sense of those numbers. When you add more data and your confidence interval shrinks to (-0.005,0.01) you can immediately see how the data adjusted and possibly supported your conclusion.

And more of a philosophical note : The real mean is most probably not zero, because the world is most probably not ideal. If you measure ton of data in the above example and get the confidence interval of (0.0001,0.0002), the null is rejected, but i assume that for all practical purposes the mean is effectively negligible.

I hope you are aware of the equivalence: null is rejected on X% confidence level <=> null value is outside of X% confidence interval.

  • $\begingroup$ So which has more power in my example, the 'test result that did not meet requirements of h0' or 'the p value which is way too high at 0.36'? Or is the only option that both have to agree to reject the null, and anything else (p value is fine but test result is not and vice versa) is a not reject? $\endgroup$ – Harry Jan 5 '15 at 12:42
  • $\begingroup$ Oh, I probably did not understand you correctly and I am not sure now, but: it cannot happen that the two do not agree. P-value is satisfied (low enough) only when the test statistic is outside the critical levels. If it does not happen for you, check your calculations. $\endgroup$ – airguru Jan 5 '15 at 13:39
  • $\begingroup$ I performed the 't-test:unequal variances' as per this link. In your opinion is this a good way to do this?. $\endgroup$ – Harry Jan 5 '15 at 13:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.