The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
ReadThe first learning rule for a linear classifier: nudge the weights after every mistake. Every network since is stacks of this unit with a smoother training signal.
My notes
Started here on purpose. It is written like psychology, not CS, which surprised me — Rosenblatt is arguing about how a brain might store information, and the learning rule falls out almost as a side effect.
The Whole Idea in One Picture
w ← w + η · (y − ŷ) · x
What Stuck
- The update only fires on mistakes. Correct predictions teach it nothing — very different from gradient descent, where every example pulls a little.
- If the data is linearly separable, it provably converges. If it is not (XOR), it never settles. That single limitation froze the field for a decade after Minsky & Papert.
- Swap the hard step for a sigmoid and you get logistic regression; stack them and add backprop and you get an MLP. Nothing conceptually new was needed, just a differentiable activation.
Why It Still Matters to Me
When I debug a classifier head in a bigger model, I still think of it as a perceptron with a better loss. The mental model of "a hyperplane that gets shoved around by errors" is the right starting intuition for almost everything later on this list.