Scenario
CBT Study: n = 40 Β· Outcome: BDI reduction (points) Β· Therapy vs. Control
Β· Ξ² = 4.80, SE = 2.10
Β· Using this one study, the tool shows in two acts which questions frequentist and Bayesian analysis can answer directly (Act 1) and how a prior as cumulative prior knowledge sharpens the posterior (Act 2).
Act 1
Same Data β different questions possible
What am I seeing?
The curve, the so-called posterior distribution of Ξ², shows which effect sizes are still plausible after taking prior knowledge (prior) and data into account. The blue area to the right of the chosen threshold shows how likely an effect above this boundary is.
What should I do?
Drag the threshold with the mouse and think about: from which effect size would the difference be substantively or clinically relevant to me? For example: should the effect be larger than 2, 2.5, or 5 BDI points?
How do I read the result?
The percentage directly answers the question: how likely is it that the true effect is larger than my chosen threshold? Example: P(Ξ² > 3) = 80% means: after taking the prior and the data into account, the probability that the effect is larger than 3 BDI points is 80%.
What's the difference to the frequentist analysis here?
A classical 95% confidence interval or a p-value does not answer this question directly. A p-value can, for example, show whether a result would be surprising under a specific null hypothesis. But it does not say: "How probable is it that the true effect is larger than 3?" With the posterior distribution, that probability can be read directly off the area under the curve.
Frequentist
Ξ² = 4.80 SE = 2.10
95% CI: [0.55, 9.05]
t(38) = 2.29 p = .028
β
P(Ξ² > 3) = ?
no direct answer
β
P(Ξ² > 0) = ?
no direct answer
β
P(2 < Ξ² < 7) = ?
no direct answer
Bayes β Posterior Distribution
(drag threshold with mouse)
β drag threshold β
Is the effect clinically meaningful
(β₯ 3.0 BDI points)?
(β₯ 3.0 BDI points)?
βFrequentist: not directly β p = .028 only says: "significant"
βP(Ξ² > 3.0) = β
What is the probability the effect
is positive at all?
is positive at all?
βp < .05 does not answer this
βP(Ξ² > 0) = β
Is the effect in the practically
relevant range [2, 7]?
relevant range [2, 7]?
βCI spans more than this range
βP(2 < Ξ² < 7) = β
Act 2
Prior as Cumulative Knowledge
What am I seeing?
Three curves: Prior (green, dashed β knowledge before the data), Likelihood (orange, dotted β what the data alone say) and Posterior (blue, filled β combined result). Below: a width comparison of 95% intervals.
What should I do?
Switch between non-informative and informative prior. The likelihood (data) stays identical β only the prior changes, and therefore the posterior.
How do I interpret this?
With an informative prior (10 prior studies: ΞΌ = 5, Ο = 0.6) the posterior narrows considerably. Same data, more precise posterior β prior knowledge formally reduces uncertainty.
Why is this useful?
Frequentism structurally ignores prior knowledge. Bayes formalises cumulative learning: today's posterior is tomorrow's prior. Scientific progress as an accumulated state of knowledge.
Note: Likelihood (orange) is constant across all priors.
The y-axis scales to the tallest curve β so it appears smaller when the prior or posterior is narrower and therefore taller.
Bayes β non-informative prior
[0.68, 8.92]
Width: 8.23 points
Bayes β informative prior (10 studies)
[3.86, 6.12]
Width: 2.26 points β 73% narrower
β
Frequentism has no formal mechanism for prior knowledge.
With an informative prior β derived here from 10 prior studies β the posterior
becomes substantially more precise: same data, narrower interval.
That is cumulative learning in science.
Note: The small difference between the frequentist CI and the Bayesian interval here must not be read as a methodological advantage of Bayes. The frequentist CI correctly uses the t-distribution because the residual spread is estimated from the data. For the Bayesian display, a normal approximation of the likelihood of Ξ² with SE = 2.10 is used for simplicity, to avoid needlessly complicating the illustration. Because of this different computation, the Bayesian interval is already marginally narrower under the non-informative prior and should not be counted as an advantage.
Further Reading
Kruschke (2014) β HDI/ROPE decision logic Β· McElreath (2020) β Bayesian workflow, prior as cumulative knowledge