Type I & II Errors and Power

Slide the effect size, sample size and α and watch the error rates and power trade off

Advanced

Testing H₀: μ = 100 against H₁: μ = 100 + δ using the mean of n observations with standard deviation σ. The curves are the sampling distributions of x̄ under each hypothesis. Drag on the chart to move the critical value.

Test
H₀: μ = 100H₁: μ = 106.0104.98090100110120130
α: Type I error β: Type II error power = 1 − β critical value
α (Type I)
0.050
β (Type II)
0.361
Power 1 − β
0.639
Standard error
3.00
Effect size d
0.40(small)
Critical z
1.645
Where the numbers come from

SE = σ/√n = 15/√25 = 3.000

critical x̄ = 100 + 1.645 × 3.000 = 104.93

power = P(Z ≥ 1.645 − 2.000) = 0.639

Power curve

against
0.00.20.40.60.81.0target power 0.8α = 0.0501255075100125150175200sample size n

Click or drag on the curve to set n.

DecisionH₀ true (μ = 100)H₀ false (μ = 106.0)
Reject H₀

Type I error

α = 0.050

false positive

Correct

power = 0.639

true positive

Fail to reject

Correct

1 − α = 0.950

true negative

Type II error

β = 0.361

false negative

Sample size calculator
Target power

n = ((z₁₋α + z₁₋β) · σ / δ)²

= ((1.645 + 0.842) × 15 / 6.0)² = 38.64

Smallest n with power ≥ 0.8: 39

Two ways to be wrong

A hypothesis test ends in one of two decisions, and the truth is one of two states, so there are four outcomes. Two of them are mistakes. A Type I error rejects a true null hypothesis; a Type II error keeps a false one.

The chart shows both at once. The blue curve is how x̄ varies if H₀ is true, and the purple curve is how it varies if the true mean is μ₀ + δ. The critical value splits the axis into "reject" and "fail to reject". The red area under the blue curve beyond it is α. The amber area under the purple curve on the wrong side is β, and the green rest of the purple curve is the power.

Move the critical value to the right and α shrinks, but β grows: for a fixed sample you cannot reduce both. The way out is to make the curves narrower. Their spread is the standard error σ/√n, so a bigger sample (or less noisy measurements) separates the curves and lowers β at the same α.

Type I error (α)

Rejecting H₀ when it is actually true: a false alarm. You set its probability directly by choosing α.

Type II error (β)

Failing to reject H₀ when it is false: a real effect goes unnoticed. Its probability depends on the effect, σ, n and α.

Power (1 − β)

The chance the test detects an effect that really exists. Studies are usually designed for a power of at least 0.8.

Sample size is the lever

More data shrinks the standard error, pulls the two curves apart and raises power without touching α.

Key formulas

Power of a one-sided z-test (H₁: μ > μ₀)

power = 1 − β = 1 − Φ(z₁₋α − δ / (σ/√n))

δ
true difference μ₁ − μ₀
z₁₋α
critical value, e.g. 1.645 for α = 0.05
Φ
standard normal CDF

Two-sided test (H₁: μ ≠ μ₀)

power = 1 − Φ(z₁₋α/₂ − δ√n/σ) + Φ(−z₁₋α/₂ − δ√n/σ)

Sample size for a target power

n = ((z₁₋α + z₁₋β) · σ / δ)², rounded up (use z₁₋α/₂ for a two-sided test)

z₁₋β
0.842 for power 0.8, 1.282 for 0.9
d = δ/σ
standardized effect size

Power for δ = 6, σ = 15

The effect is d = 6/15 = 0.4 standard deviations. Power rises with n and falls when α is made stricter or the test is two-sided.
nSE = σ/√nOne-sided, α = 0.05Two-sided, α = 0.05One-sided, α = 0.01
104.740.3520.2440.144
253.000.6390.5160.372
502.120.8820.8070.692
1001.500.9910.9790.953

How to use this visualizer

  1. 1
    Start with the defaults: δ = 6, σ = 15, n = 25, α = 0.05, one-sided. Read α, β and power from the tiles and the coloured areas.
  2. 2
    Drag on the chart to move the critical value. Watch the red area (α) and the amber area (β) trade off against each other.
  3. 3
    Increase n. Both curves get narrower, the overlap shrinks and power rises while α stays fixed.
  4. 4
    Increase σ or decrease δ. The curves overlap more and power drops. Set δ = 0 and power equals α.
  5. 5
    Switch the power curve between sample size and effect δ. Click on the curve to jump to a point.
  6. 6
    Use the sample size calculator to find the smallest n that reaches power 0.8, then press Set n to check it on the chart.

Common misconception

"Fail to reject H₀" does not mean "there is no effect". With a small sample the power can be low: at δ = 6, σ = 15 and n = 10, a one-sided test at α = 0.05 misses this real effect about 65% of the time. A non-significant result from an underpowered study is weak evidence either way.

Practice problems

Problem 1: Power of a one-sided test

Test H₀: μ = 100 against H₁: μ > 100 with σ = 15, n = 36 and α = 0.05. If the true mean is 106, what are the critical value for x̄, the power and β?
Show solution

SE = 15/√36 = 2.5

Critical x̄ = 100 + 1.645 × 2.5 = 104.11. Reject H₀ if x̄ ≥ 104.11.

If μ = 106: z = (104.11 − 106)/2.5 = −0.755

Power = P(Z ≥ −0.755) = Φ(0.755) ≈ 0.775, so β ≈ 0.225

Critical x̄ ≈ 104.11; power ≈ 0.775; β ≈ 0.225

Problem 2: Planning a study

You want a two-sided test at α = 0.05 to detect a difference of 4 units when σ = 10, with power 0.9. How many observations do you need?
Show solution

z₁₋α/₂ = z₀.₉₇₅ = 1.960 and z₁₋β = z₀.₉₀ = 1.282

n = ((1.960 + 1.282) × 10 / 4)² = (3.242 × 2.5)² = 8.104² ≈ 65.7

Round up to 66. Check: with n = 66, SE = 10/√66 = 1.231, δ/SE = 3.250, power = 1 − Φ(1.960 − 3.250) + Φ(−5.210) ≈ 0.901.

n = 66 observations

Related topics

Discussion

Questions, ideas or a different approach? Join the conversation.