Linear Model — Bayes Thinking Lab

Step 1

Draw data points — click in the field below to place points. Imagine you are measuring the relationship between two variables: e.g. study hours (x) and test score (y). Place 8–15 points.

Data points

Count0

x-range—

y-range—

Display

Show residuals Conditional distributions (OLS only)

What is a regression line?

A regression line describes the linear relationship between two variables. It answers: if x increases by 1 — how much does y change on average?

The line has two parameters: the intercept a (where does the line cross the y-axis when x=0?) and the slope b (how steep is the line?). Formula: ŷ = a + b · x

Why is it useful? Because it allows computing a predicted value ŷ for any x — even for values not in the data. It also summarizes the relationship in two interpretable numbers: Is there a relationship at all (b ≠ 0)? And how strong is it?

Why the "best" line? There are infinitely many possible lines — the OLS line is the one that is closest on average to all data points. Try to find it yourself: any other line will have a higher RSS value.

What are residuals?

No data point lies exactly on the line — the deviation between the predicted value (point on the line) and the actual value is called a residual.

Residuals are "what the model does not explain". A good line has small residuals — it lies as close as possible to all points.

Residuals are visualized as the red vertical lines.

RSS — how do we measure fit?

To combine all residuals into a single measure of fit, they are squared and summed: RSS = Σ(y − ŷ)²

Why square? So that positive and negative residuals do not cancel out. Large deviations are penalized more heavily than small ones.

The OLS line is precisely the line that minimizes RSS — that is the meaning of "Ordinary Least Squares".

Conditional normal distributions — why this matters

The linear model assumes: the y-values around each x-value are normally distributed — with mean on the regression line and constant spread σ.

Not the overall distribution of y needs to be normal — only the spread around the line. This is a common misconception.

Toggle "Conditional distributions" — you will see small bell curves at different x positions, all centered on the line.

ℹ Linear Model — Help

What will I learn?

What does it mean to fit a line through data points — and why is exactly one line the "best"? This tool answers that in three steps:

What is a regression line — and how do you read a and b?
What are residuals — and why do we square them (RSS)?
How does OLS minimize RSS — and what does that mean geometrically?
What are conditional normal distributions along the line?

Step by step

Draw data points — click in the field. 8–15 points are enough.
Fit the line yourself — adjust a and b with the sliders. Watch the RSS: when does it get smaller?
Reveal OLS — click "★ Show best OLS line". Your blue line remains visible: compare it directly with the optimal one.

Tip: enable Conditional distributions in the OLS step — the small bell curves show what the model assumes about the spread.

What do the statistics mean?

RSS (Residual Sum of Squares) = Σ(y − ŷ)² — sum of squared deviations. OLS minimizes this value exactly.

R² — proportion of variance in y explained by x. R²=1: perfect prediction. R²=0: the line explains nothing.

σ — estimated spread of residuals. In a Bayesian model, σ corresponds to the prior on the error distribution: y ~ Normal(μ, σ).

Why this matters for Bayes

The linear model is the starting point for everything that follows. In the Bayesian context: OLS is equivalent to maximum likelihood under a normal distribution. Once priors are added, this becomes Bayesian regression — with the same residual concept.

Next → Maximum Likelihood: why OLS and MLE yield the same result under normality