1.3 Confidence intervals and two-sided tests | Inferential Statistics | Comparing two groups | UvA

Test statistics and p-values are useful for determining statistical significance, but small effects can appear significant with large sample sizes, making practical significance hard to assess. Confidence intervals offer an alternative by providing a range of plausible values for a given statistic.

A 95% confidence interval indicates that if samples were repeatedly drawn, 95% of those intervals would contain the true population parameter. It's crucial to understand that it does not mean there's a 95% chance that a single interval contains the population value; it either does or it doesn't. Confidence intervals facilitate the interpretation of practical significance based on their width and proximity of the null hypothesis parameter.

Confidence intervals are based on the sampling distribution. The boundaries are determined by the sample statistic, the test statistic value (T or Z), and the standard error. With repeated sampling, 95% of these intervals will contain the true population parameter. If the null hypothesis value lies outside the confidence interval, it suggests that the null hypothesis is unlikely to be true, leading to its rejection.

Two-sided tests and confidence intervals are closely related, with decisions based on whether boundary values are exceeded. They are complementary; a two-sided test aims for a test statistic outside the critical region, while a confidence interval aims for the null hypothesis value outside its boundaries. The margin calculations for both are similar but expressed in different units.