Simple Perceptron
The perceptron is the fundamental building block of neural networks. Watch it learn to classify data points by adjusting its weights and bias.
The Perceptron
A perceptron is the simplest type of artificial neuron. It takes inputs, multiplies them by weights, adds a bias, and produces an output through an activation function.
output = activate(w₁·x + w₂·y + b)
The perceptron learns by adjusting weights and bias to minimize prediction errors.
Visualization
Parameters
Training Statistics
How the Perceptron Learns
Understanding the perceptron
What is a perceptron?
A perceptron is a simple artificial neuron that takes multiple inputs, applies weights to them, adds a bias, and produces a binary output (0 or 1). It was invented by Frank Rosenblatt in 1957 and is the foundation of modern neural networks.
How does it learn?
The perceptron uses supervised learning. For each training example:
- 1It makes a prediction based on the current weights.
- 2It compares the prediction with the actual label.
- 3It calculates the error (the difference between prediction and actual).
- 4It updates the weights proportionally to the error and learning rate.
Key components
Weights (w₁, w₂)
Bias (b)
Activation function
Learning rate
Mathematical formulation
Forward pass
z = w₁ × x + w₂ × y + b
output = activate(z) = { 1 if z ≥ 0, 0 otherwise }Weight update rule
error = actual_label - predicted_label w₁ = w₁ + α × error × x w₂ = w₂ + α × error × y b = b + α × error
(where α is the learning rate)
Limitations of the perceptron
Only linearly separable data
A single perceptron can only classify data that can be separated by a straight line (or a hyperplane in higher dimensions).Cannot solve the XOR problem
The famous XOR problem cannot be solved by a single perceptron, which led to the development of multi-layer networks.Solution: multi-layer perceptrons
By stacking multiple layers of perceptrons with non-linear activations, we can solve complex non-linear problems.Historical significance
1957: Frank Rosenblatt invented the perceptron at the Cornell Aeronautical Laboratory.
1958: The Mark I Perceptron was the first machine to recognize simple shapes and patterns, sparking excitement about artificial intelligence.
1969: Minsky and Papert's book "Perceptrons" highlighted its limitations, causing the first "AI Winter."
1980s: The development of backpropagation and multi-layer networks overcame these limitations, leading to modern deep learning.
Experiment ideas
- 1Try different learning rates and observe how quickly the perceptron converges.
- 2Generate new data and see how the perceptron adapts to different distributions.
- 3Manually adjust weights to understand how they affect the decision boundary.
- 4Watch how the bias parameter shifts the decision line up or down.
Discussion
Questions, ideas or a different approach? Join the conversation.