Law of Large Numbers
Flip coins and roll dice thousands of times and watch the running frequency settle on the true probability
Experiment
Running frequency of heads(blue band: p ± 2√(p(1 − p)/n), where about 95% of runs should be)
Last 0 outcomes
Outcome counts(bars: observed share, black line: true probability)
Results
- Trials n
- 0
- Count of heads
- 0
- Relative frequency
- –
- True probability p
- 0.5000(1/2)
- Error |f − p|
- –
- 95% band ±
- –
Early on the line swings wildly: one extra success out of 10 trials moves the frequency by 0.1. After 1,000 trials one extra success moves it by only 0.001, so the line flattens out near p. The count of successes still wanders away from p × n; it is the proportion that settles.
What the law says
Repeat a random experiment many times, independently and under the same conditions, and count how often an event happens. The law of large numbers says that the relative frequency of the event approaches its true probability p as the number of trials n grows. More generally, the average of n independent values from the same distribution approaches the distribution's mean.
This is what connects probability to the real world. A probability of 1/6 for rolling a six does not predict any single roll, but it does predict that in 6,000 rolls you will see close to 1,000 sixes, and the larger the number of rolls the closer the proportion gets.
The law says nothing about how fast that happens. For that you need the spread of the relative frequency, √(p(1 − p)/n), and its shape, which the central limit theorem says is approximately normal for large n.
Relative frequency
Convergence
Proportions, not counts
No memory
Key formulas
Relative frequency
fn = X / n
- X
- number of times the event happened in n trials
- f
- relative frequency after n trials
Mean and spread of the relative frequency
E[fn] = p, SD(fn) = √(p(1 − p) / n) ≤ 1 / (2√n)
- p
- true probability of the event on each trial
Weak law of large numbers
for every ε > 0: P(|fn − p| > ε) → 0 as n → ∞
Rough 95% band
p ± 2√(p(1 − p) / n), which is never wider than p ± 1/√n
How fast does a fair coin settle?
| Flips n | SD of proportion | About 95% of runs end between |
|---|---|---|
| 10 | 0.158 | 0.18 to 0.82 |
| 100 | 0.050 | 0.40 to 0.60 |
| 1,000 | 0.0158 | 0.468 to 0.532 |
| 10,000 | 0.0050 | 0.490 to 0.510 |
| 100,000 | 0.00158 | 0.4968 to 0.5032 |
Each extra digit of accuracy costs 100 times as many trials, because the error shrinks with √n, not n.
How to use this visualizer
- 1Pick an experiment. For the biased coin set P(heads) with the slider; for the die choose which face to track. Two dice tracks a sum of 7, which has probability 6/36.
- 2Press +1 a few times and watch the running frequency jump around. Then press +100 or +1000, or Auto-run, and watch the line settle on the dashed true probability.
- 3Switch the log scale on and off. On a linear axis the early chaos is squeezed into the left edge; the log scale gives every factor of 10 the same width.
- 4Turn on Overlay 5 independent runs. Each run wanders differently at first, but all of them end up inside the shrinking blue band.
- 5Compare the bars of observed outcome shares with the black true-probability lines below the chart. With more trials the bars line up with the lines.
The gambler's fallacy
The law of large numbers does not mean that a run of heads makes tails "due". Each flip is still 50/50. The proportion returns toward 0.5 because the early streak becomes a smaller and smaller part of a growing total, not because later flips make up for it.Practice problems
Problem 1: How many flips?
Show solutionHide solution
SD of the proportion: √(p(1 − p)/n) = √(0.25 / 100) = 0.5 / 10 = 0.05
95% band: 0.5 ± 2 × 0.05 = 0.40 to 0.60
For SD = 0.01: √(0.25 / n) = 0.01, so 0.25 / n = 0.0001 and n = 2,500
Between 0.40 and 0.60 heads; n = 2,500 flips
Problem 2: A streak gets diluted
Show solutionHide solution
The coin has no memory, so P(heads on flip 11) = 0.5.
Expected heads in the remaining 990 flips: 0.5 × 990 = 495
Expected total: 10 + 495 = 505 heads out of 1,000
Expected proportion: 505 / 1000 = 0.505. The streak is not cancelled out; its effect on the proportion is just diluted, from 1.0 to about 0.505.
P(heads on flip 11) = 0.5; expected proportion ≈ 0.505
Related topics
Discussion
Questions, ideas or a different approach? Join the conversation.