Key takeaways
- Linear regression predicts a continuous outcome; logistic (logit) regression predicts a binary outcome such as pass/fail.
- Read output in three layers: overall model fit (R², F), individual coefficients (b, SE, t, p, CI) and diagnostics.
- A coefficient’s t statistic is b divided by its standard error; |t| above about 2 usually means p < .05 in reasonable samples.
- Check linearity, independence, homoscedasticity, normal residuals and multicollinearity before trusting the numbers.
- Write up model fit first, then each predictor with b, SE (or CI), t and p, and say what the coefficients mean in plain language.
What is regression analysis?
Regression analysis models the relationship between an outcome (the dependent variable) and one or more predictors (independent variables), so you can estimate how much the outcome changes for each unit change in a predictor while holding the others constant. Unlike correlation, which treats two variables symmetrically, regression has a direction: it predicts Y from X.
The general linear model is Ŷ = b₀ + b₁X₁ + b₂X₂ + … + bₖXₖ, where b₀ is the intercept (predicted Y when all predictors are zero) and each b is a slope.
Which type of regression do you need?
| Type | Outcome | Use when |
|---|---|---|
| Simple linear regression | Continuous | One predictor; you want the slope and prediction, not just the correlation |
| Multiple linear regression | Continuous | Several predictors, or one predictor with controls |
| Hierarchical regression | Continuous | You enter predictors in blocks to see what each block adds (ΔR²) |
| Moderation analysis | Continuous (usually) | You test whether the effect of X depends on a third variable, by adding an X × moderator interaction term |
| Binary logistic regression | Two categories (yes/no) | Predicting the odds of an event, such as passing or dropping out |
| Multinomial or ordinal logistic | Three or more categories | Unordered or ordered categorical outcomes |
Logit vs logistic regression: they are the same model. “Logit” refers to the link function, the log of the odds, ln[p / (1 − p)], which logistic regression models as a linear function of the predictors.
Logistic regression vs chi-square: a chi-square test checks whether two categorical variables are associated. Logistic regression can do the same for a binary outcome but also handles continuous predictors, several predictors at once, and gives adjusted odds ratios. With a single categorical predictor and no controls, chi-square is simpler; otherwise logistic regression is usually the better choice. Our guide to choosing a statistical test sets this in context.
Simple linear regression: a worked example
Using five students’ weekly revision hours (X: 2, 4, 5, 6, 8) and exam scores (Y: 55, 62, 60, 71, 77), with means x̄ = 5 and ȳ = 65:
- Slope:
b₁ = Σ[(x − x̄)(y − ȳ)] / Σ(x − x̄)²= 75 / 20 = 3.75. - Intercept:
b₀ = ȳ − b₁x̄= 65 − 3.75 × 5 = 46.25. - Equation: Ŷ = 46.25 + 3.75X.
Each extra weekly hour of revision is associated with a 3.75-point higher predicted score. A student revising 7 hours has a predicted score of 46.25 + 26.25 = 72.5.
The standard error of the estimate (the typical size of a residual) is sₑ = √[ Σ(y − ŷ)² / (n − k − 1) ]. Here the residual sum of squares is 32.75, so sₑ = √(32.75 / 3) ≈ 3.30 points. R² = 1 − 32.75/314 ≈ .90, which equals the squared correlation in simple regression.
How to read regression output
Suppose a researcher predicts job satisfaction (1–7 scale) from supervisor support, pay satisfaction and years of tenure in 120 employees. The output has three parts.
1. Model summary. R² = .32: the three predictors together account for 32% of the variance in satisfaction. Adjusted R² = .30 corrects for the number of predictors and is the better estimate for the population.
2. ANOVA table. F(3, 116) = 18.20, p < .001. The model predicts better than the mean alone. Degrees of freedom are k = 3 and n − k − 1 = 116.
3. Coefficients table.
| Predictor | b | SE | t | p | 95% CI for b |
|---|---|---|---|---|---|
| Supervisor support | 0.48 | 0.11 | 4.36 | < .001 | [0.26, 0.70] |
| Pay satisfaction | 0.21 | 0.09 | 2.33 | .021 | [0.03, 0.39] |
| Tenure (years) | −0.06 | 0.08 | −0.75 | .455 | [−0.22, 0.10] |
- b (unstandardised coefficient): holding the other predictors constant, each one-point increase in supervisor support is associated with a 0.48-point increase in satisfaction.
- t stat:
t = b / SE, tested on n − k − 1 degrees of freedom. For support, 0.48 / 0.11 ≈ 4.36. There is no universal “good” t; |t| above roughly 1.98 is significant at .05 here, but significance is not the same as importance. - β (standardised coefficient): the change in Y in standard deviation units per standard deviation change in X. Useful for comparing predictors on different scales.
- CI: tenure’s interval includes zero, matching its non-significant p.
Want a second pair of eyes? Running a regression for your thesis? We can review your model and output with you and explain each coefficient so you can defend it. See dissertation statistics help.
Assumptions to check
Most regression assumptions concern the residuals (observed minus predicted values), not the raw variables.
- Linearity: the relationship between each predictor and the outcome is linear. Check a plot of residuals against predicted values for curves.
- Independence of errors: one case’s residual does not predict another’s. A design issue; the Durbin–Watson statistic helps with time-ordered data.
- Homoscedasticity: residuals have roughly constant spread across predicted values. A funnel shape suggests a problem; robust (heteroscedasticity-consistent) standard errors are one fix.
- Normality of residuals: mainly matters for small samples. Check a histogram or Q–Q plot of residuals.
- No severe multicollinearity: predictors are not near-duplicates. Variance inflation factors (VIF) above about 5–10 are a warning sign.
- No overly influential cases: check Cook’s distance and investigate values well above the rest.
Logistic regression drops the normality and homoscedasticity assumptions but requires linearity between continuous predictors and the log odds, independent observations, little multicollinearity and enough cases in the smaller outcome category.
Logistic regression: interpreting odds ratios
In logistic regression, coefficients are on the log-odds scale, so they are usually exponentiated into odds ratios (OR = exp(b)). An OR above 1 means higher odds of the outcome; below 1 means lower odds.
Example: predicting whether students pass a statistics module (yes/no) from attendance at optional workshops (number attended), controlling for prior grade. The workshop coefficient is b = 0.65, SE = 0.21. Then:
- OR = exp(0.65) ≈ 1.92: each additional workshop is associated with nearly double the odds of passing.
- 95% CI = exp(0.65 ± 1.96 × 0.21) = [1.27, 2.89].
- Wald χ²(1) = (0.65 / 0.21)² ≈ 9.58, p = .002.
Remember that odds are not probabilities: doubling the odds does not double the chance of passing. Report model fit with a likelihood-ratio χ² test against the null model and, if you like, a pseudo-R² such as Nagelkerke’s, clearly labelled.
Common regression mistakes
- Too many predictors for the sample. Adding variables always raises R², even when they add nothing real. Choose predictors from theory and prior research, and watch adjusted R².
- Stepwise selection as the main strategy. Letting software pick predictors by p value produces unstable models and biased coefficients. Prefer theory-driven or hierarchical entry.
- Uncentred interaction terms. In moderation analysis, the main effects in a model with an interaction are effects when the other variable equals zero. Centring continuous predictors makes them interpretable.
- Comparing raw coefficients across scales. A b of 0.48 for a 1–7 scale and 0.02 for income in dollars are not comparable; use standardised coefficients or meaningful units.
- Extrapolating. Predictions outside the range of your data, such as for 20 revision hours when nobody studied more than 8, are guesses.
How to write up regression results in APA style
Describe the model, report overall fit, then the predictors, and finish with a plain-language interpretation. Put full coefficients in a table when there are more than two or three predictors.
A multiple linear regression was conducted to predict job satisfaction from supervisor support, pay satisfaction and tenure. The model explained a significant proportion of variance, R² = .32, adjusted R² = .30, F(3, 116) = 18.20, p < .001. Supervisor support was a significant positive predictor, b = 0.48, 95% CI [0.26, 0.70], t(116) = 4.36, p < .001, as was pay satisfaction, b = 0.21, 95% CI [0.03, 0.39], t(116) = 2.33, p = .021. Tenure was not a significant predictor, b = −0.06, 95% CI [−0.22, 0.10], t(116) = −0.75, p = .455.
Logistic regression showed that each additional workshop attended was associated with higher odds of passing the module, b = 0.65, SE = 0.21, OR = 1.92, 95% CI [1.27, 2.89], p = .002, controlling for prior grade.
Mention assumption checks briefly in the text (“Residual plots indicated no violations of linearity or homoscedasticity; all VIFs were below 2”). Avoid causal language unless your design supports it: write “was associated with” or “predicted”, not “caused”. For hands-on help building and explaining models in your own data, see our dissertation statistics help or Stata help.