Neural Tangent Kernel: Convergence and Generalization in Neural Networks

Why this one

Read this after Support-Vector Networks and Understanding deep learning requires rethinking generalization. The useful idea is not that real networks are secretly kernel machines. It is that there is a regime where training a huge network can be described as moving a function by a fixed similarity measure between examples. That lets you replace the usual muddy story, millions of parameters rolling downhill, with a cleaner one: which directions in the data does gradient descent fit first, and which does it fit slowly? People get this paper wrong when they treat NTK as an explanation of all deep learning. It is more valuable as a controlled limit case, a ruler you can hold next to finite networks to see what changes when features are learned rather than frozen. If you build or tune models, the evening pays off because it makes early stopping, width, kernels, and optimization geometry feel like one subject instead of four.