Significance Levels and p - Values

When conducting a hypothesis test, the criterion for rejecting thenull hypothesis is that an observed value is so unlikely assumingthat the null hypothesis is true that it must in fact be true. Todecide this, each hypothesis test is associated with a significancelevel and the observed value must be less likely than thesignificance leveltypically1% or 5%. When the null hypothesis is rejected, the result is said tobe statistically significant.

If the distribution is a continuous distribution, then thesignificance level is the level of the test. A problem arises fordiscrete distributions, because it is usually impossible to obtain asignificance level of exactly 1% or 5% or whatever level is required.For example, suppose we assume a binomial distributionandwe are required to conduct a hypothesis test at the 10% level. Werequire the probabilities contained in the upper and lower tails tobe 5% each. From the cumulative binomial distribution tables thelower end gives

and

We choose the greatest value less than 0.05 ie 0.0416.

At the upper end,andWechoose the first, this being that closest to but less than 0.05.

The total significance level is then 0.0416+0.048=0.0896.

On the other hand the–value is the probability of observing a value at least as unlike asone that is actually observed, assuming the null hypothesis is true..For example, suppose we assume the distributionasbefore, and we observe 10 successes. Then the-value is

The significance level for a test conducted assuming a continuousdistribution is always at least equal to the significance level of atest conducted using a discrete distribution.