ImageNet Classification with Deep Convolutional Neural Networks
Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton2012NeurIPS 2012
Read it on doi.org(opens in a new tab)Why this one
Read this before Deep Residual Learning for Image Recognition and Batch Normalization, because AlexNet is the baseline those papers are trying to make less painful. The lesson is not just that convolutional nets beat hand-built vision features. It is that a pile of practical choices, ReLUs, dropout, data augmentation, GPU splitting, careful preprocessing, and enough labeled data, can move a model family from toy to dominant. People often retell this as a single algorithmic breakthrough, but the useful reading is more operational: what changed when the authors treated training speed, regularization, and hardware limits as first-class design constraints. If you are building models, this is worth an evening because it shows deep learning becoming an engineering discipline, not a theorem with a demo attached.
What to take away
- ReLUs mattered because they made a large CNN train fast enough to iterate on real ImageNet data.
- Dropout and image augmentation were not decoration, they kept a huge supervised model from simply memorizing.
- The two-GPU architecture is a reminder that model design often bends around hardware before hardware catches up.
Reads with
- Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
makes alexnet-style deep convnet training faster and less brittle
- Deep Residual Learning for Image Recognition
starts from alexnet's imagenet win and removes the depth barrier
- Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
turns alexnet's relu choice into a principled initialization rule