The p-value is the probability of obtaining a test statistic that is equal to or more extreme than the one observed, assuming that the null hypothesis is true. It is used in hypothesis testing to determine the significance of the observed results.
For example, if the null hypothesis is that the population mean is equal to a specific value, and the observed sample means significantly different from that value, the p-value would be the probability of obtaining a sample mean that is that different or more different from the population mean, assuming that the null hypothesis is true.
If the p-value is less than the significance level of the test, the null hypothesis is rejected, and the alternative hypothesis is accepted. The null hypothesis is not rejected (fail to reject) if the p-value is greater than or equal to the significance level.
For example, if the p-value is 0.045 and the significance level is 0.05, the null hypothesis can be rejected. This is because the p-value is less than the significance level, indicating that the observed results are unlikely to have occurred by chance if the null hypothesis is true.
An Example of Using the P-Value
All statistical software provide the p-value as an output of any hypothesis test. To evaluate the result of any hypothesis test, the p-value can be used to conclude if you would reject or fail to reject the test. For example, the following was the output using Excel in a two-sample z-test conducted to confirm if the two populations have an equal mean.
In this test, although two samples have slightly different means (150.6139 and 150.1138), the p-value here is 0.1373. This value is more than 0.05. That means with 95% confidence we fail to reject the null hypothesis. We would conclude that this difference between sample means is just a random difference, and we do not have sufficient evidence to say that these two means are different.
Summary
In summary, the p-value is the probability of obtaining a test statistic that is equal to or more extreme than the one observed, assuming that the null hypothesis is true. It is used in hypothesis testing to determine the significance of the observed results. It is compared to the significance level of the test to determine whether to reject or fail to reject the null hypothesis.