Simple Perceptron

The perceptron is the fundamental building block of neural networks. Watch it learn to classify data points by adjusting its weights and bias.

Beginner

The Perceptron

A perceptron is the simplest type of artificial neuron. It takes inputs, multiplies them by weights, adds a bias, and produces an output through an activation function.

output = activate(w₁·x + w₂·y + b)

The perceptron learns by adjusting weights and bias to minimize prediction errors.

Visualization

Class 0 (Below line)
Class 1 (Above line)
Misclassified

Parameters

Training Statistics

Epoch
0
Accuracy
0.0%
Data Points
0
Status
Ready

How the Perceptron Learns

1
Forward Pass: Calculate output = activate(w₁·x + w₂·y + b)
2
Calculate Error: error = actual_label - predicted_label
3
Update Weights: w₁ = w₁ + learning_rate × error × x
4
Repeat: Continue until accuracy is satisfactory

Understanding the perceptron

What is a perceptron?

A perceptron is a simple artificial neuron that takes multiple inputs, applies weights to them, adds a bias, and produces a binary output (0 or 1). It was invented by Frank Rosenblatt in 1957 and is the foundation of modern neural networks.

How does it learn?

The perceptron uses supervised learning. For each training example:

  1. 1
    It makes a prediction based on the current weights.
  2. 2
    It compares the prediction with the actual label.
  3. 3
    It calculates the error (the difference between prediction and actual).
  4. 4
    It updates the weights proportionally to the error and learning rate.

Key components

Weights (w₁, w₂)

Control how much each input influences the output.

Bias (b)

Shifts the decision boundary, allowing the model to fit better.

Activation function

Converts the weighted sum to a binary output (step function).

Learning rate

Controls how much the weights change with each update.

Mathematical formulation

Forward pass

z = w₁ × x + w₂ × y + b
output = activate(z) = { 1 if z ≥ 0, 0 otherwise }

Weight update rule

error = actual_label - predicted_label
w₁ = w₁ + α × error × x
w₂ = w₂ + α × error × y
b  = b  + α × error

(where α is the learning rate)

Limitations of the perceptron

Only linearly separable data

A single perceptron can only classify data that can be separated by a straight line (or a hyperplane in higher dimensions).

Cannot solve the XOR problem

The famous XOR problem cannot be solved by a single perceptron, which led to the development of multi-layer networks.

Solution: multi-layer perceptrons

By stacking multiple layers of perceptrons with non-linear activations, we can solve complex non-linear problems.

Historical significance

1957: Frank Rosenblatt invented the perceptron at the Cornell Aeronautical Laboratory.

1958: The Mark I Perceptron was the first machine to recognize simple shapes and patterns, sparking excitement about artificial intelligence.

1969: Minsky and Papert's book "Perceptrons" highlighted its limitations, causing the first "AI Winter."

1980s: The development of backpropagation and multi-layer networks overcame these limitations, leading to modern deep learning.

Experiment ideas

  1. 1
    Try different learning rates and observe how quickly the perceptron converges.
  2. 2
    Generate new data and see how the perceptron adapts to different distributions.
  3. 3
    Manually adjust weights to understand how they affect the decision boundary.
  4. 4
    Watch how the bias parameter shifts the decision line up or down.

Discussion

Questions, ideas or a different approach? Join the conversation.