1.4 Power of a hypothesis test | Inferential Statistics | Comparing two groups | UvA

Power is the probability of correctly rejecting the null hypothesis, meaning detecting a hypothesized effect if it truly exists. It's crucial to distinguish between correct decisions, Type I errors (false positives, rejecting a true null, probability alpha), and Type II errors (false negatives, failing to reject a false null, probability beta).

Alpha (significance level) is the probability of a Type I error. The complement, 1-alpha, is the confidence level. Beta is the probability of a Type II error. Power is 1-beta, the probability of correctly rejecting the null hypothesis. Lowering alpha increases beta and thus reduces power, while increasing alpha increases power. These are inversely related.

Power can be increased by increasing the number of observations (sample size), which narrows confidence intervals and makes test statistics larger. It can also be increased by reducing standard deviation through using reliable measurement instruments, eliminating random variability, or selecting a more homogeneous sample.

Another way to increase power is to obtain a larger sample statistic value, which can be achieved through stronger experimental manipulation or selecting a sample where a stronger effect is expected. Using one-sided tests instead of two-sided tests also increases power, as does using parametric tests over nonparametric tests, given their assumptions are met.

Post-hoc power estimation occurs after data collection, using the sample statistic as a proxy for the true population value. A priori power calculations, performed before data collection, are more valuable. They involve estimating the sample statistic using standard effect sizes and then calculating the required sample size to achieve a desired power level (e.g., 0.8).

A priori power calculations are crucial for designing effective research studies and selecting appropriate statistical tests. They help determine if a study is worthwhile by ensuring a high enough probability of detecting a true effect, justifying the effort and resources invested.