On this page
Choosing a test is a matter of answering four questions about your design. Answer them in order and the test is usually determined before you open any software.
- What type is my outcome variable — continuous, ordinal, binary or count?
- How many groups or predictors am I comparing?
- Are the observations independent, or repeated on the same people?
- Am I comparing groups, testing a relationship, or predicting an outcome?
Comparing groups
| Groups | Independent | Repeated / paired | Non-parametric alternative |
|---|---|---|---|
| Two | Independent t-test | Paired t-test | Mann-Whitney U / Wilcoxon signed-rank |
| Three or more | One-way ANOVA | Repeated-measures ANOVA | Kruskal-Wallis / Friedman |
| Two factors | Two-way ANOVA | Mixed ANOVA | Aligned rank transform |
| With a covariate | ANCOVA | Mixed model | — |
| Several outcomes | MANOVA | Repeated MANOVA | — |
Testing relationships
- Two continuous variables, linear relationship — Pearson correlation.
- Ordinal data, or a non-linear monotonic relationship — Spearman rho.
- Two categorical variables — chi-square test of independence, or Fisher's exact test when expected counts are small.
- Agreement between two raters — Cohen's kappa, not correlation.
- Internal consistency of a scale — Cronbach's alpha or McDonald's omega.
Predicting an outcome
| Outcome variable | Model |
|---|---|
| Continuous | Multiple linear regression |
| Binary (yes/no) | Binary logistic regression |
| Three or more unordered categories | Multinomial logistic regression |
| Ordered categories | Ordinal logistic regression |
| Counts | Poisson or negative binomial regression |
| Time until an event | Cox proportional hazards |
| Nested or clustered data | Multilevel / mixed-effects model |
When assumptions fail
A violated assumption is a fork in the road, not a dead end. There are four normal responses, and all of them are reportable.
- Use a robust variant — Welch's t-test and Welch's ANOVA do not assume equal variances and cost you very little when variances are equal anyway.
- Bootstrap — resampling gives confidence intervals without distributional assumptions, and most software offers it as a checkbox.
- Transform — log or square-root transformation can fix skew, but interpretation is then on the transformed scale, which you must state.
- Switch to a non-parametric test — reliable, though usually with some loss of power, and the hypothesis being tested subtly changes.
Whichever you choose, report the violation and the response. A reader who sees that you noticed and handled it trusts the rest of the analysis more, not less.
Three errors that recur constantly
- Running many t-tests instead of an ANOVA. Six pairwise comparisons at α = .05 gives roughly a 26% chance of at least one false positive.
- Treating a Likert item as continuous. A single item is ordinal. A summed scale of several items is conventionally treated as continuous, and that distinction matters.
- Testing significance instead of the hypothesis. Decide what result would support or contradict your hypothesis before you run anything.
Frequently asked questions
A single Likert item is ordinal and is best analysed as such. A scale summing several items measuring one construct is conventionally treated as continuous, and that is widely accepted.
Unequal group sizes are fine for most tests, but they make the homogeneity-of-variance assumption more consequential. Welch's correction handles it well.
Need expert help with this?
Send the brief and the deadline. You get a reference number straight away and a fixed quote, usually within one working day.
Get Expert Help Prefer to talk it through? Ask on WhatsApp