Building a Perceptron from Scratch: Neural Networks Stripped to One Weight and a Bias

A tutorial walks through implementing Frank Rosenblatt’s 1958 perceptron in plain Python, framing it as the minimal unit underlying every modern neural network. The classifier takes a single input, multiplies it by a weight, adds a bias, and outputs a binary decision based on whether the result exceeds zero. Learning happens through a simple update loop: when the prediction is wrong, the weight and bias get nudged by an amount proportional to the error and a learning rate, repeated across full passes of the data called epochs.

The piece uses two toy problems to expose why each component matters. Classifying numbers as positive or negative works fine without bias because the natural boundary sits at zero. But asking whether an exam score passes (threshold 50) traps a bias-less perceptron at roughly 50 percent accuracy, because a line forced through the origin cannot separate values that all sit on the positive side of zero. The bias term lets the decision boundary slide to wherever the real cutoff lives.

The author also illustrates why input normalization matters: since weight updates scale with the input value, large raw numbers cause erratic jumps during training. Rescaling inputs to a tidy range smooths convergence and, more importantly, prevents features measured on large scales (like salary in dollars) from drowning out small-scale binary features (like a yes/no flag) when multiple inputs are combined.