Law of Large Numbers

Flip coins and roll dice thousands of times and watch the running frequency settle on the true probability

Beginner

Experiment

Running frequency of heads(blue band: p ± 2√(p(1 − p)/n), where about 95% of runs should be)

00.250.50.751p = 0.500110number of trials n (log scale)Run some trials to draw the running frequency

Last 0 outcomes

No trials yet

Outcome counts(bars: observed share, black line: true probability)

HT

Results

Trials n
0
Count of heads
0
Relative frequency
–
True probability p
0.5000(1/2)
Error |f − p|
–
95% band ±
–

Early on the line swings wildly: one extra success out of 10 trials moves the frequency by 0.1. After 1,000 trials one extra success moves it by only 0.001, so the line flattens out near p. The count of successes still wanders away from p × n; it is the proportion that settles.

What the law says

Repeat a random experiment many times, independently and under the same conditions, and count how often an event happens. The law of large numbers says that the relative frequency of the event approaches its true probability p as the number of trials n grows. More generally, the average of n independent values from the same distribution approaches the distribution's mean.

This is what connects probability to the real world. A probability of 1/6 for rolling a six does not predict any single roll, but it does predict that in 6,000 rolls you will see close to 1,000 sixes, and the larger the number of rolls the closer the proportion gets.

The law says nothing about how fast that happens. For that you need the spread of the relative frequency, √(p(1 − p)/n), and its shape, which the central limit theorem says is approximately normal for large n.

Relative frequency

After n trials, the relative frequency of an event is the number of times it happened divided by n. It is a random number that changes with every trial.

Convergence

As n grows, the relative frequency gets closer and closer to the true probability p. The typical error shrinks like 1/√n.

Proportions, not counts

The gap between the count and its expected value p × n usually grows (like √n). Only the proportion settles down.

No memory

Coins and dice do not remember past outcomes. Early streaks are not corrected; they are diluted by the many trials that follow.

Key formulas

Relative frequency

fn = X / n

X
number of times the event happened in n trials
f
relative frequency after n trials

Mean and spread of the relative frequency

E[fn] = p,   SD(fn) = √(p(1 − p) / n) ≤ 1 / (2√n)

p
true probability of the event on each trial

Weak law of large numbers

for every ε > 0:  P(|fn − p| > ε) → 0  as n → ∞

Rough 95% band

p ± 2√(p(1 − p) / n),  which is never wider than p ± 1/√n

How fast does a fair coin settle?

For p = 0.5 the standard deviation of the proportion is 0.5/√n. About 95% of runs end inside the band p ± 2 SD.
Flips nSD of proportionAbout 95% of runs end between
100.1580.18 to 0.82
1000.0500.40 to 0.60
1,0000.01580.468 to 0.532
10,0000.00500.490 to 0.510
100,0000.001580.4968 to 0.5032

Each extra digit of accuracy costs 100 times as many trials, because the error shrinks with √n, not n.

How to use this visualizer

  1. 1
    Pick an experiment. For the biased coin set P(heads) with the slider; for the die choose which face to track. Two dice tracks a sum of 7, which has probability 6/36.
  2. 2
    Press +1 a few times and watch the running frequency jump around. Then press +100 or +1000, or Auto-run, and watch the line settle on the dashed true probability.
  3. 3
    Switch the log scale on and off. On a linear axis the early chaos is squeezed into the left edge; the log scale gives every factor of 10 the same width.
  4. 4
    Turn on Overlay 5 independent runs. Each run wanders differently at first, but all of them end up inside the shrinking blue band.
  5. 5
    Compare the bars of observed outcome shares with the black true-probability lines below the chart. With more trials the bars line up with the lines.

The gambler's fallacy

The law of large numbers does not mean that a run of heads makes tails "due". Each flip is still 50/50. The proportion returns toward 0.5 because the early streak becomes a smaller and smaller part of a growing total, not because later flips make up for it.

Practice problems

Problem 1: How many flips?

A fair coin is flipped 100 times. Between which two proportions of heads will about 95% of such experiments land? How many flips would you need for the standard deviation of the proportion to be 0.01?
Show solution

SD of the proportion: √(p(1 − p)/n) = √(0.25 / 100) = 0.5 / 10 = 0.05

95% band: 0.5 ± 2 × 0.05 = 0.40 to 0.60

For SD = 0.01: √(0.25 / n) = 0.01, so 0.25 / n = 0.0001 and n = 2,500

Between 0.40 and 0.60 heads; n = 2,500 flips

Problem 2: A streak gets diluted

A fair coin lands heads on each of its first 10 flips. You then flip it 990 more times. What is the probability of heads on flip 11, and what is the expected proportion of heads after all 1,000 flips?
Show solution

The coin has no memory, so P(heads on flip 11) = 0.5.

Expected heads in the remaining 990 flips: 0.5 × 990 = 495

Expected total: 10 + 495 = 505 heads out of 1,000

Expected proportion: 505 / 1000 = 0.505. The streak is not cancelled out; its effect on the proportion is just diluted, from 1.0 to about 0.505.

P(heads on flip 11) = 0.5; expected proportion ≈ 0.505

Related topics

Discussion

Questions, ideas or a different approach? Join the conversation.