Hypothesis Testing

Run a z-test step by step and see the test statistic, rejection region and p-value

Intermediate

A machine should fill bottles with 500 ml on average. An inspector weighs a sample of n bottles and tests whether the mean fill is off target.

Test
H₁
α

Distribution of z if H₀ is true (standard normal)

-1.9601.960z = 1.50p = 0.1336-4-3-2-101234
rejection region (area α = 0.05) p-value area observed z
z statistic
1.500
p-value
0.1336
Critical value
±1.960
Decision
Keep H₀
Step by step
  1. 1
    Hypotheses. H₀: μ = 500 ml (the machine is on target). H₁: μ ≠ 500 ml. Significance level α = 0.05.
  2. 2
    Standard error. SE = σ/√n = 4.00/√25 = 0.800
  3. 3
    Test statistic. z = (x̄ − μ₀)/SE = (501.20 − 500)/0.800 = 1.500
  4. 4
    p-value. p = 2 · P(Z ≥ 1.500) = 0.1336
  5. 5
    Decision. p > α (0.1336 > 0.05), equivalently the rejection rule |z| ≥ 1.960 is not met, so we fail to reject H₀. There is not enough evidence that the mean fill differs from 500 ml. This does not prove that μ = 500.
Simulate the inspection

Bottles are drawn from a normal distribution with this mean and σ = 4 ml.

Draw one sample to fill in x̄ above, or run 1,000 tests to see how often this test rejects H₀ in the chosen world.

How a hypothesis test works

A hypothesis test is a proof by contradiction with uncertainty. You start from a null hypothesis H₀ that pins down the population mean, say μ = 500 ml, and an alternative H₁ that says what you suspect instead: the mean is different, lower or higher.

Assuming H₀ is true, the sample mean x̄ should land near μ₀, with a spread given by the standard error. The test statistic z = (x̄ − μ₀)/(σ/√n) measures how far away it actually landed, in standard errors. Under H₀ it follows a standard normal distribution, so you can ask how unusual the observed value is. That probability is the p-value.

If the p-value is at most the significance level α, the data would be rare under H₀, so you reject H₀. Otherwise you fail to reject it, which is not the same as showing it is true. When σ is unknown you use the sample standard deviation s instead, and the statistic follows a t distribution with n − 1 degrees of freedom.

Null hypothesis H₀

The "nothing is going on" claim, such as "the machine fills 500 ml on average". The test assumes it is true and looks for evidence against it.

Test statistic

How many standard errors the sample mean is from the value claimed by H₀. Large values in the direction of H₁ count against H₀.

p-value

The probability, if H₀ were true, of a test statistic at least as extreme as the one observed. Small p means the data would be surprising under H₀.

Decision at level α

Reject H₀ when p ≤ α. The significance level α is chosen before looking at the data and is the false-alarm rate you accept.

Key formulas

One-sample z statistic (σ known)

z = (x̄ − μ₀) / (σ / √n)

x̄
sample mean
μ₀
mean claimed by H₀
σ / √n
standard error of x̄

One-sample t statistic (σ unknown)

t = (x̄ − μ₀) / (s / √n)

s
sample standard deviation; compare with a t distribution with n − 1 df

Decision rule

reject H₀ if p-value ≤ α   (equivalently, if the statistic falls in the rejection region)

p-values and rejection regions

For a t-test, replace Z with T on n − 1 degrees of freedom and use t critical values.
TestH₁p-valueReject at α = 0.05
Two-sidedμ ≠ μ₀2 · P(Z ≥ |z|)|z| ≥ 1.960
Left-tailedμ < μ₀P(Z ≤ z)z ≤ −1.645
Right-tailedμ > μ₀P(Z ≥ z)z ≥ 1.645

How to use this visualizer

  1. 1
    Choose the test (z if σ is known, t if not), the alternative hypothesis and the significance level α.
  2. 2
    Move the sample mean slider. The black line is the observed statistic, the orange area is its p-value and the red area is the rejection region.
  3. 3
    Read the step-by-step panel: it fills in the standard error, the statistic, the p-value and the decision with the current numbers.
  4. 4
    Increase n with the same x̄. The standard error shrinks, the statistic grows and a small difference becomes significant.
  5. 5
    In the simulation panel, set the true mean to 500 ml (H₀ true) and run 1,000 tests. About α of them reject H₀: these are Type I errors.
  6. 6
    Move the true mean away from 500 and run again. Now the rejection rate is the power of the test, and the misses are Type II errors.

Common misconception

The p-value is not the probability that H₀ is true. It is computed assuming H₀ is true, and tells you how surprising the data would be in that case. A p-value of 0.03 means "data this extreme would happen 3% of the time if H₀ held", not "there is a 3% chance H₀ is right". Likewise, a large p-value only says the data are compatible with H₀, not that H₀ has been proven.

Practice problems

Problem 1: Is the filling machine on target?

A machine should fill bottles with 500 ml; σ = 4 ml is known. A sample of 36 bottles has mean 498.6 ml. Test H₀: μ = 500 against H₁: μ ≠ 500 at α = 0.05. Would the decision change at α = 0.01?
Show solution

SE = σ/√n = 4/√36 = 4/6 = 0.667 ml

z = (498.6 − 500)/0.667 = −1.4/0.667 = −2.10

p = 2 · P(Z ≤ −2.10) = 2 × 0.0179 = 0.0357

At α = 0.05: 0.0357 ≤ 0.05, so reject H₀. At α = 0.01: 0.0357 > 0.01, so fail to reject.

z = −2.10, p ≈ 0.036: reject at α = 0.05, but not at α = 0.01

Problem 2: Battery life claim

A maker claims its batteries last at least 20 hours. Ten batteries last a mean of 19.1 hours with s = 1.5 hours. Test H₀: μ = 20 against H₁: μ < 20 at α = 0.05.
Show solution

σ is unknown, so use a t-test with n − 1 = 9 degrees of freedom.

SE = s/√n = 1.5/√10 = 0.474 hours

t = (19.1 − 20)/0.474 = −1.897

Critical value: t₀.₀₅(9) = −1.833. Since −1.897 ≤ −1.833, the statistic is in the rejection region.

p = P(T₉ ≤ −1.897) ≈ 0.045 ≤ 0.05

t ≈ −1.90, p ≈ 0.045: reject H₀; the data suggest the batteries last less than 20 hours

Related topics

Discussion

Questions, ideas or a different approach? Join the conversation.