Type I & II Errors and Power
Slide the effect size, sample size and α and watch the error rates and power trade off
Testing H₀: μ = 100 against H₁: μ = 100 + δ using the mean of n observations with standard deviation σ. The curves are the sampling distributions of x̄ under each hypothesis. Drag on the chart to move the critical value.
- α (Type I)
- 0.050
- β (Type II)
- 0.361
- Power 1 − β
- 0.639
- Standard error
- 3.00
- Effect size d
- 0.40(small)
- Critical z
- 1.645
SE = σ/√n = 15/√25 = 3.000
critical x̄ = 100 + 1.645 × 3.000 = 104.93
power = P(Z ≥ 1.645 − 2.000) = 0.639
Power curve
Click or drag on the curve to set n.
| Decision | H₀ true (μ = 100) | H₀ false (μ = 106.0) |
|---|---|---|
| Reject H₀ | Type I error α = 0.050 false positive | Correct power = 0.639 true positive |
| Fail to reject | Correct 1 − α = 0.950 true negative | Type II error β = 0.361 false negative |
n = ((z₁₋α + z₁₋β) · σ / δ)²
= ((1.645 + 0.842) × 15 / 6.0)² = 38.64
Smallest n with power ≥ 0.8: 39
Two ways to be wrong
A hypothesis test ends in one of two decisions, and the truth is one of two states, so there are four outcomes. Two of them are mistakes. A Type I error rejects a true null hypothesis; a Type II error keeps a false one.
The chart shows both at once. The blue curve is how x̄ varies if H₀ is true, and the purple curve is how it varies if the true mean is μ₀ + δ. The critical value splits the axis into "reject" and "fail to reject". The red area under the blue curve beyond it is α. The amber area under the purple curve on the wrong side is β, and the green rest of the purple curve is the power.
Move the critical value to the right and α shrinks, but β grows: for a fixed sample you cannot reduce both. The way out is to make the curves narrower. Their spread is the standard error σ/√n, so a bigger sample (or less noisy measurements) separates the curves and lowers β at the same α.
Type I error (α)
Type II error (β)
Power (1 − β)
Sample size is the lever
Key formulas
Power of a one-sided z-test (H₁: μ > μ₀)
power = 1 − β = 1 − Φ(z₁₋α − δ / (σ/√n))
- δ
- true difference μ₁ − μ₀
- z₁₋α
- critical value, e.g. 1.645 for α = 0.05
- Φ
- standard normal CDF
Two-sided test (H₁: μ ≠ μ₀)
power = 1 − Φ(z₁₋α/₂ − δ√n/σ) + Φ(−z₁₋α/₂ − δ√n/σ)
Sample size for a target power
n = ((z₁₋α + z₁₋β) · σ / δ)², rounded up (use z₁₋α/₂ for a two-sided test)
- z₁₋β
- 0.842 for power 0.8, 1.282 for 0.9
- d = δ/σ
- standardized effect size
Power for δ = 6, σ = 15
| n | SE = σ/√n | One-sided, α = 0.05 | Two-sided, α = 0.05 | One-sided, α = 0.01 |
|---|---|---|---|---|
| 10 | 4.74 | 0.352 | 0.244 | 0.144 |
| 25 | 3.00 | 0.639 | 0.516 | 0.372 |
| 50 | 2.12 | 0.882 | 0.807 | 0.692 |
| 100 | 1.50 | 0.991 | 0.979 | 0.953 |
How to use this visualizer
- 1Start with the defaults: δ = 6, σ = 15, n = 25, α = 0.05, one-sided. Read α, β and power from the tiles and the coloured areas.
- 2Drag on the chart to move the critical value. Watch the red area (α) and the amber area (β) trade off against each other.
- 3Increase n. Both curves get narrower, the overlap shrinks and power rises while α stays fixed.
- 4Increase σ or decrease δ. The curves overlap more and power drops. Set δ = 0 and power equals α.
- 5Switch the power curve between sample size and effect δ. Click on the curve to jump to a point.
- 6Use the sample size calculator to find the smallest n that reaches power 0.8, then press Set n to check it on the chart.
Common misconception
"Fail to reject H₀" does not mean "there is no effect". With a small sample the power can be low: at δ = 6, σ = 15 and n = 10, a one-sided test at α = 0.05 misses this real effect about 65% of the time. A non-significant result from an underpowered study is weak evidence either way.Practice problems
Problem 1: Power of a one-sided test
Show solutionHide solution
SE = 15/√36 = 2.5
Critical x̄ = 100 + 1.645 × 2.5 = 104.11. Reject H₀ if x̄ ≥ 104.11.
If μ = 106: z = (104.11 − 106)/2.5 = −0.755
Power = P(Z ≥ −0.755) = Φ(0.755) ≈ 0.775, so β ≈ 0.225
Critical x̄ ≈ 104.11; power ≈ 0.775; β ≈ 0.225
Problem 2: Planning a study
Show solutionHide solution
z₁₋α/₂ = z₀.₉₇₅ = 1.960 and z₁₋β = z₀.₉₀ = 1.282
n = ((1.960 + 1.282) × 10 / 4)² = (3.242 × 2.5)² = 8.104² ≈ 65.7
Round up to 66. Check: with n = 66, SE = 10/√66 = 1.231, δ/SE = 3.250, power = 1 − Φ(1.960 − 3.250) + Φ(−5.210) ≈ 0.901.
n = 66 observations
Related topics
Discussion
Questions, ideas or a different approach? Join the conversation.