1.2 P-values | Inferential Statistics | Comparing two groups | UvA

The null hypothesis is essential for determining the test statistic distribution and calculating probabilities. With a null and alternative hypothesis, we calculate the p-value: the probability of observing the same or a more extreme test statistic, assuming the null hypothesis is true. This video will explain how to determine and use p-values with significance levels to reject the null hypothesis.

When comparing two groups, like raw meat vs. canned food for cat health, the alternative hypothesis determines if we perform a one-sided or two-sided test. A one-sided test is used when we expect a specific directional difference (e.g., raw meat is healthier), calculating the p-value from one tail of the distribution. A two-sided test is used when we expect a difference but no specific direction, doubling the probability from either tail.

To decide whether to reject the null hypothesis, the p-value is compared to the significance level (alpha), a pre-set value representing the risk of mistakenly rejecting a true null hypothesis. The most common alpha is 0.05, though it's an arbitrary choice. If the p-value is less than or equal to alpha, we reject the null hypothesis; otherwise, we fail to reject it. We never 'accept' the null hypothesis.

While statistical software now calculates p-values, historical methods involved using tables. These tables list critical boundary values for test statistics at common significance levels. If an observed test statistic falls in the critical region (more extreme than the critical boundary value), it means the p-value is smaller than the significance level, allowing for rejection of the null hypothesis.

Hypothesis testing requires certain assumptions to be met, particularly for parametric tests like Z, T, and F tests. These assumptions concern the distribution of the population. If assumptions are not met, the shape of the test statistic distribution may be unknown, leading to incorrect p-value estimation and potentially erroneous conclusions. Always verify the assumptions of any statistical test you use.