Object Detection

Follow a detector's raw output through score filtering and non-maximum suppression, compute IoU by dragging boxes, and see how a YOLO grid divides up the work.

Advanced

Simulated detections

No neural network runs on this page. The candidate boxes and confidence scores are generated deterministically around the objects in a drawn scene, to look like what a detector outputs before post-processing. Everything that happens to those boxes (score filtering, IoU, NMS, matching, precision and recall) is computed for real.

Ground truth (dotted)

Score threshold0.30NMS IoU threshold0.50

23 of 24 candidate boxes pass the score threshold. Press Step to start NMS.

Candidates (best score first)

tree0.95waiting
person0.93waiting
car0.91waiting
dog0.90waiting
car0.77waiting
dog0.77waiting
person0.73waiting
tree0.70waiting
person0.64waiting
car0.63waiting
dog0.62waiting
tree0.60waiting
dog0.58waiting
car0.52waiting
tree0.52waiting
person0.51waiting
dog0.48waiting
tree0.44waiting
dog0.43waiting
person0.35waiting
tree0.35waiting
car0.34waiting
person0.31waiting
car0.22below score

Precision and recall (after full NMS)

A detection counts as correct if it has the right class and IoU ≥ 0.5 with an unmatched object.

	TP	FP	FN	Precision	Recall
Before NMS	4	19	0	0.17	1.00
After NMS	4	4	0	0.50	1.00

Duplicates count as false positives, which is why NMS raises precision. Raising the score threshold trades recall for precision.

The candidate boxes and scores are simulated: generated deterministically around each object to mimic raw detector output. The IoU, NMS and precision/recall computations on them are real.

How object detection works

Classification answers what is in an image. Detection also answers where, with a bounding box for every object. A trained detector such as YOLO or Faster R-CNN produces hundreds of candidate boxes, each with a class and a confidence score. Many of them cover the same object, slightly shifted. Post-processing turns that crowd into one box per object.

Boxes plus scores

A detector outputs many boxes, each with a class and a confidence score. Most are duplicates or noise.

IoU measures overlap

Intersection over union is 1 for identical boxes and 0 for disjoint ones. It is used both to match and to de-duplicate.

NMS removes duplicates

Keep the best box, remove its same-class neighbours that overlap too much, repeat.

Grids make it one pass

YOLO splits the image into cells and predicts boxes for every cell at once, which makes it fast.

IoU calculator

Intersection over union is the overlap measure behind both NMS and evaluation.

Drag a box to move it. Drag its white corner handle to resize.

Area A: 160 × 120 = 19200
Area B: 170 × 120 = 20400
Intersection: 70 × 70 = 4900
Union: 19200 + 20400 − 4900 = 34700

IoU = intersection / union

4900 / 34700 = 0.141

Below 0.5: usually counted as a miss, and NMS at threshold 0.5 would keep both boxes.

The YOLO grid

YOLO divides the image into S × S cells. The cell that contains an object's centre is responsible for predicting it.

Grid size S5 × 5

Output tensor (B = 2 boxes per cell, C = 4 classes)

S × S × (B·5 + C) = 5 × 5 × 14 = 350 numbers

Tap a grid cell to see what it is trained to predict.

Object	Cell	x, y in cell	w, h
car	(3, 1)	0.25, 0.69	0.31, 0.23
person	(3, 2)	0.27, 0.20	0.09, 0.42
tree	(2, 3)	0.79, 0.63	0.23, 0.65
dog	(3, 4)	0.73, 0.91	0.10, 0.14

x and y are the object centre relative to its cell (0 to 1); w and h are relative to the whole image. If two centres land in the same cell, that cell has to predict both, which is one reason YOLO uses several boxes per cell (and later versions use anchors).

Key formulas

Intersection over union

IoU(A, B) = |A ∩ B| / |A ∪ B| = |A ∩ B| / (|A| + |B| − |A ∩ B|)

A, B: two boxes
|·|: area

Precision and recall

precision = TP / (TP + FP), recall = TP / (TP + FN)

TP: detections matched to an object (right class, IoU ≥ 0.5)
FP: unmatched detections, including duplicates
FN: objects that no detection matched

YOLO output size

S × S × (B · 5 + C)

B: boxes per cell, each with x, y, w, h, objectness
C: number of classes

Detector families

Model	Stages	Core idea	Speed
R-CNN / Fast R-CNN	Two	Propose ~2000 regions, then classify and refine each one	Slow
Faster R-CNN	Two	A Region Proposal Network shares features with the classifier	Medium
YOLO	One	Grid cells predict boxes, objectness and classes in a single forward pass	Fast
SSD / RetinaNet	One	Anchor boxes at several feature-map scales; focal loss for class imbalance	Fast
DETR	One	A transformer predicts a fixed set of boxes, matched one-to-one, so no NMS needed	Medium

How to use this visualizer

1
Look at the scene: dotted black boxes are the ground truth, coloured boxes are the simulated candidates.
2
Set the score threshold. Candidates below it disappear before NMS even starts.
3
Press Step. The best remaining box is kept (filled) and every same-class box whose IoU with it exceeds the NMS threshold is suppressed (grey, dashed).
4
Press Play or Finish to run to the end, then compare precision and recall before and after NMS.
5
Lower the IoU threshold to suppress more aggressively; raise it and duplicates survive as false positives.
6
Drag the two boxes in the IoU calculator, and tap cells in the YOLO grid to see what each cell is trained to predict.

Code

JavaScript

function iou(a, b) {
  const x1 = Math.max(a.x, b.x), y1 = Math.max(a.y, b.y);
  const x2 = Math.min(a.x + a.w, b.x + b.w), y2 = Math.min(a.y + a.h, b.y + b.h);
  const inter = Math.max(0, x2 - x1) * Math.max(0, y2 - y1);
  return inter / (a.w * a.h + b.w * b.h - inter);
}

function nms(boxes, scoreThreshold, iouThreshold) {
  let remaining = boxes
    .filter((b) => b.score >= scoreThreshold)
    .sort((a, b) => b.score - a.score);
  const kept = [];
  while (remaining.length) {
    const best = remaining.shift();
    kept.push(best);
    remaining = remaining.filter((b) => b.cls !== best.cls || iou(best, b) <= iouThreshold);
  }
  return kept;
}

Practice problems

Problem 1: Compute IoU

Box A spans x 0 to 10 and y 0 to 10. Box B spans x 5 to 15 and y 5 to 15. What is their IoU? Would NMS with a threshold of 0.5 suppress B if A has the higher score?

Show solution

Intersection: x 5 to 10, y 5 to 10, so 5 × 5 = 25.

Union: 100 + 100 − 25 = 175.

IoU = 25 / 175 ≈ 0.143.

0.143 ≤ 0.5, so B is kept.

Problem 2: Run NMS by hand

Three "dog" boxes have scores 0.9 (P), 0.8 (Q) and 0.6 (R). IoU(P, Q) = 0.7, IoU(P, R) = 0.2 and IoU(Q, R) = 0.6. With an IoU threshold of 0.5, which boxes survive?

Show solution

Step 1: keep P (highest score). Q has IoU 0.7 > 0.5 with P, so Q is suppressed. R has 0.2, so it stays.

Step 2: keep R. Nothing remains.

P and R survive.

Q never gets to suppress R, because Q was removed first. Order matters in greedy NMS.

Discussion

Questions, ideas or a different approach? Join the conversation.