From LM to GLM — Bayes Thinking Lab

Three Scenarios

OLS regression assumes normally distributed errors (ε). Three scenarios from body image research show what happens when you force an LM anyway — and how a GLM with an appropriate distribution resolves the situation.

What is this about?

A linear regression (LM) assumes that the errors are normally distributed — the outcome may then take any value. This does not fit when the outcome is only 0/1 (therapy success), a count (body checking per week) or a positive, right-skewed quantity (reaction time). The LM then predicts values that cannot exist (probabilities below 0, negative counts), and the residuals do not look normal.

The GLM solves this with three building blocks: a distribution family that matches the data, a linear predictor η and a link function that maps η into the valid range. The LM is the special case with a normal distribution and identity link.

The normality assumption concerns the errors ε, not the outcome itself — with these data it is violated structurally, not just by chance.

How to proceed

First: Pick one of the three scenarios above (Bernoulli, Poisson, Gamma) and read the "The problem" box.
① LM (left): The LM is fitted despite unsuitable data. The red box below the plot names the specific problems.
② GLM (middle): The same data with a suitable distribution and link — logit for Bernoulli, log link for Poisson and Gamma. The green box explains why it works here.
③ Fit comparison (right): Residual distribution, AIC and RMSE of both models. ΔAIC = AIC(LM) − AIC(GLM): the larger, the clearer the GLM.

What to watch for

• AIC is a decision aid, not a verdict: ΔAIC < 2 means hardly any difference, ΔAIC > 10 a clear advantage for the GLM.
• RMSE is given in y units and is always comparable — with AIC it pays to look at both numbers.
• For count data Poisson is not always enough: with overdispersion you switch to the negative binomial distribution.

Things to try

• Set the sample size n to 20 and click "New data" several times: how stable is ΔAIC?
• Switch between the scenarios and compare the shape of the residuals in the right plot.
• Look at the link function further down: it shows how η is mapped into the valid range.

What's next?

How the distribution of an outcome shifts with the predictor value is shown in GLM Cond Dist.

Scenarios – Please pick one: ↓

Problem

①LM — Normal distribution (wrong)

→

②GLM — appropriate distribution (right)

→

③Fit comparison: AIC & residuals

Sample size n 50

LM with Normal distribution ⚠ Problematic

Log-Likelihood (LM)—

AIC (LM)—

GLM with appropriate distribution ✓ Better

Log-Likelihood (GLM)—

AIC (GLM)—

Residual distribution & AIC comparison

ΔAIC (LM − GLM)—

■ LM-AIC —

■ GLM-AIC —

          RMSE (always comparable — in y-units; smaller = better)
        

■ LM-RMSE —

■ GLM-RMSE —

Link Function What the link function does — visually

    How to read this diagram: On top is the linear predictor η = a + b·x —
    just like in ordinary linear regression, it can take any value between −∞ and +∞.
    The link function (arrow) maps η onto the valid range for this distribution family
    (below) — e.g. probabilities in (0,1) for Bernoulli, or positive values for Poisson/Gamma.
    The five colored points are the same example x-values on top and below — the
    connecting lines show directly how the link function compresses or stretches them
    to bring them into the valid range.
  

      Linear predictor η = a + b·x
       · Value range: −∞ to +∞
    

↓

      E[y] = g⁻¹(η)
      Value range: restricted
    

Worked Example Effect interpretation with concrete numbers

Concepts What you learn here

How do you choose the right family?

Look at the nature of the outcome variable — range, discrete/continuous, symmetric/skewed:

Bernoulli — binary outcomes: diagnoses, success/failure.
Poisson — count data (0, 1, 2, …). Note: with overdispersion → Negative Binomial.
Gamma — positive, right-skewed: reaction times, waiting times, costs.
Normal/LM — continuous, symmetric, no hard zero boundary: z-standardised questionnaire scores, IQ.

Check fit via Posterior Predictive Check · AIC/BIC for comparing alternative families.

Go through it systematically: The Likelihood Compass walks through all distribution families — with required parameters, priors, and brms syntax for each.

What is a link function — and why do you need one?

The problem: The linear predictor η = a + b·x can take any value (−∞ to +∞). But many outcome variables have a restricted range: probabilities lie in (0,1), count rates must be positive.

The solution: A link function transforms the expected value into a range where linear modelling makes sense:

Logit link (Bernoulli): log(p/(1-p)) = η — maps p ∈ (0,1) to (−∞,+∞); inversely, every η gives a p = 1/(1+e^(−η)) ∈ (0,1). The S-curve in the plot.
Log link (Poisson, Gamma): log(λ) = η — maps λ > 0 to (−∞,+∞); inversely, λ = e^η is always positive. The exponential curve in the plot.
Identity link (Normal): E[y] = η — no transformation; the LM is a GLM special case with normal distribution and identity link.

What happens when you choose the wrong likelihood?

The LM minimises RSS — which is equivalent to maximum-likelihood estimation under a normal distribution assumption. When this assumption is violated, the estimates can still be computed, but:

Predictions fall outside the valid range (p < 0 or p > 1)
The log-likelihood is suboptimal — AIC/BIC worse
Residuals are systematically non-normal
Inference (confidence intervals, tests) is biased

ΔAIC as a decision guide

AIC penalises poor fit and number of parameters: AIC = −2ℓ̂ + 2k

Rule of thumb for ΔAIC = AIC(LM) − AIC(GLM):

ΔAIC < 2: barely any difference
ΔAIC 2–6: moderate advantage for GLM
ΔAIC > 10: strong advantage, LM clearly worse

Here you see the ΔAIC live. With binary data and count data the advantage of the GLM is typically large — with continuous, symmetric data the LM may be sufficient.