ImageNet Classification with Deep Convolutional Neural Networks
ReadA deep convolutional network trained on GPUs cut ImageNet error nearly in half. It shows why depth, ReLU, and data augmentation became the default recipe.
My notes
Less a new idea than a new recipe that finally worked at scale. Reading it now, the engineering section (two GPUs, split model) feels more important than the architecture.
The Recipe That Stuck
- ReLU instead of tanh — trained several times faster.
- GPUs — the model literally did not fit on one card.
- Data augmentation — random crops, flips, colour jitter.
- Dropout in the fully connected layers (the Dropout paper, read earlier, is the same group).
Top-5 error 15.3% vs 26.2% for the runner-up. That gap is why everyone switched.