Summary
Highlights
The selection of a statistical test depends on the research question, the scale of data, assumptions for parametric tests, experimental design, and the number of groups. The first step is to decide if the aim is to compare groups or investigate relationships between variables.
After deciding on the research aim, the next step is to determine the scale of the measured variables: continuous (numeric, measured by instruments), ordinal (categories with a natural order, e.g., good, better, best), or nominal (categories without a natural order, e.g., species A, B, C).
For continuous variables, check assumptions for parametric tests, primarily normal distribution. Parametric tests have higher statistical power if assumptions are met. If data is skewed, highly skewed, includes extreme values, or if the sample size is small (<30) for skewed data, non-parametric tests are more appropriate as they are more robust.
The study design can be unpaired (independent groups) or paired (measurements on the same individuals or matched individuals). Examples include comparing different fertilizer groups (unpaired) or 'before and after' measurements (paired).
For continuous data with fulfilled normality assumptions: use a paired t-test for two paired measurements, repeated measures ANOVA for more than two paired measurements. For unpaired designs: an unpaired t-test (independent samples t-test) for two independent groups, or a one-way ANOVA for more than two independent groups.
When normality assumptions are not met, or for ordinal data: use a Wilcoxon signed-rank test for paired continuous data (if differences are symmetric), or a sign test for ordinal data or non-symmetric continuous data. For more than two paired measurements, use the Friedman test. For unpaired designs: the Wilcoxon-Mann-Whitney test (Mann-Whitney U-test) for two independent groups, or the Kruskal-Wallis test for more than two independent groups.
For nominal scale data (proportions or relationships): For unpaired designs, use a chi-square test of homogeneity (for comparing proportions) or the two-proportion Z-test. For investigating relationships between two nominal variables (e.g., smoking and lung cancer), use the chi-squared test of independence. Fisher's exact test is an alternative if expected frequencies in contingency table cells are less than five. For paired nominal data, use McNemar's test, or Cochrane's Q test for more than two treatments.
To analyze relationships between two continuous variables, use Pearson correlation (if assumptions are fulfilled). Linear regression can predict one variable based on another. If Pearson assumptions are not met, or if one/both variables are ordinal, use Spearman correlation.
For a single sample: A one-sample t-test compares a sample mean to a population or hypothesized mean (continuous data, normality fulfilled). If normality is not met or data is ordinal, use the one-sample Wilcoxon signed-rank test or sign test. For comparing a sample proportion to a hypothesized proportion, use a one-proportion Z-test. For comparing an observed frequency distribution to a known/hypothesized distribution, use the chi-square goodness-of-fit test.
Examples include using an unpaired t-test for comparing white blood cell concentration between healthy and diseased individuals, a sign test for paired plant growth data on an ordinal scale, a one-way ANOVA or Kruskal-Wallis for fish weight in three different lakes, a chi-square test for the relationship between smoking and lung cancer, and Pearson/Spearman correlation for blood pressure and BMI.
The selection of tests can sometimes vary based on expert opinion, especially between parametric and non-parametric options. Some tests can be interchangeable, e.g., simple linear regression with a two-group numeric independent variable yields the same p-value as an unpaired t-test. This guide covers basic tests, and more complex regression models are becoming popular for controlling external factors.