Linear Regression

Place points and see the least-squares line, residuals and R² update live

Intermediate
Presets:
Tool:
00224466881010xy

Tap empty space to add a point. Drag a point to move it.

Least-squares line
ŷ = 0.837x + 1.206
Points (n)
9
Slope (b)
0.837
Intercept (a)
1.206
Pearson r
0.996
R²
0.992
SSE (best)
0.33
best fit above line below line

What is linear regression?

Linear regression finds the straight line ŷ = bx + a that best describes how one variable y changes with another variable x. "Best" has a precise meaning: the line minimises the sum of the squared vertical distances (residuals) between the points and the line. This is the method of least squares.

Squaring the residuals makes every error positive and punishes big misses much more than small ones. That is why a single outlier can drag the whole line towards it: load the "With an outlier" preset and watch the slope and R² collapse.

Residuals

A residual is the vertical gap between a data point and the line: observed y minus predicted ŷ. Points above the line have positive residuals, points below have negative ones.

Least squares

Square every residual and add them up to get the SSE. The least-squares line is the unique line that makes this sum as small as possible.

Pearson r

A number from −1 to 1 measuring how tightly the points cluster around a straight line, and in which direction. r = 0 means no linear relationship.

R² (coefficient of determination)

The fraction of the variation in y that the line explains. For simple linear regression R² = r². An R² of 0.9 means 90% of the spread in y is accounted for by x.

Key formulas

Slope and intercept

b = Sxy / Sxx    a = ȳ − b·x̄

x̄, ȳ
the means of the x and y values
Sxx
Σ(x − x̄)², the spread of x
Sxy
Σ(x − x̄)(y − ȳ), how x and y vary together

Correlation and goodness of fit

r = Sxy / √(Sxx·Syy)    R² = 1 − SSE / Syy = r²

Syy
Σ(y − ȳ)², the total variation in y
SSE
Σ(y − ŷ)², the variation the line leaves unexplained

The line always passes through the mean

Because a = ȳ − b·x̄, the point (x̄, ȳ) is always on the least-squares line. The residuals also always add up to zero.

How to use this visualizer

  1. 1
    Pick a preset or press Clear and tap the plot to place your own points.
  2. 2
    Drag any point and watch the indigo least-squares line, slope, intercept, r and R² update instantly.
  3. 3
    Switch to Remove and tap points to delete them.
  4. 4
    Turn on "My guess line" and drag the orange handles. Try to get your SSE down to the minimum; you can match it, but never beat it.
  5. 5
    Load "With an outlier" and remove the stray point to see how strongly one point can bend the fit.

Computing the line in code

Python
def least_squares(xs, ys):
    n = len(xs)
    mean_x = sum(xs) / n
    mean_y = sum(ys) / n
    sxx = sum((x - mean_x) ** 2 for x in xs)
    sxy = sum((x - mean_x) * (y - mean_y) for x, y in zip(xs, ys))
    slope = sxy / sxx
    intercept = mean_y - slope * mean_x
    return slope, intercept

Practice problems

Problem 1: Fit three points

Find the least-squares line through (1, 2), (2, 3) and (3, 5), then compute SSE and R².
Show solution

x̄ = 2, ȳ = 10/3 ≈ 3.333

Sxx = 1 + 0 + 1 = 2

Sxy = (−1)(−4/3) + 0 + (1)(5/3) = 3

b = 3 / 2 = 1.5, a = 10/3 − 1.5 × 2 = 1/3

ŷ = 1.5x + 0.333 → predictions 1.833, 3.333, 4.833

Residuals 0.167, −0.333, 0.167 → SSE = 1/6 ≈ 0.167

Syy = 16/9 + 1/9 + 25/9 = 42/9 ≈ 4.667

R² = 1 − 0.167 / 4.667 ≈ 0.964

Problem 2: Line from summary statistics

For a data set, x̄ = 10, ȳ = 40, the standard deviations are sₓ = 2 and sᵧ = 5, and r = 0.8. Find the regression line, predict y at x = 12, and interpret R².
Show solution

Slope b = r · sᵧ / sₓ = 0.8 × 5 / 2 = 2

Intercept a = ȳ − b·x̄ = 40 − 2 × 10 = 20

ŷ = 2x + 20, so at x = 12: ŷ = 44

R² = 0.8² = 0.64: the line explains 64% of the variation in y.

Discussion

Questions, ideas or a different approach? Join the conversation.