Few papers have been cited as often as Deep Residual Learning for Image Recognition. Published in December 2015 by Kaiming He, Xiangyu Zhang, Shaoqing Ren and Jian Sun at Microsoft Research, it introduced residual networks (ResNets) and a deceptively simple idea — the skip connection — that now appears in almost every deep learning model, including the transformers behind large language models.

The puzzle: deeper got worse

By 2015, deep convolutional networks were winning image recognition competitions, and deeper networks generally did better. But researchers hit a wall. Beyond a certain depth, adding more layers made networks less accurate — even on their training data.

This was not overfitting, where a model memorizes training data and fails on new data. The deeper networks were simply harder to optimize. The authors called it the degradation problem. It was strange: in theory, a deeper network could always copy a shallower one and set the extra layers to do nothing. In practice, training couldn’t find that solution.

The idea: learn the difference

ResNet changes what each block of layers has to learn. Instead of transforming its input x into a completely new output, a residual block learns a residual — a correction — and adds it back to the input:

output = x + F(x)

The “+ x” is the skip connection (also called a shortcut). The input flows around the block unchanged and is added to whatever the block computes.

Why does this help? If the best thing a block can do is nothing, it only needs to push F(x) toward zero — much easier than learning an exact identity mapping through several nonlinear layers. And during training, gradients can flow backward through the shortcuts directly, avoiding the signal decay that plagued very deep networks.

The results

The improvements were striking.

  • The team trained networks with 152 layers on ImageNet — eight times deeper than the influential VGG networks — while keeping computational cost manageable.
  • An ensemble of ResNets achieved a 3.57% top-5 error on the ImageNet test set and won first place in the ILSVRC 2015 classification task.
  • ResNets also won the 2015 competitions for ImageNet detection and localization, and for COCO detection and segmentation.
  • In experiments on the smaller CIFAR-10 dataset, the authors trained networks with over 1,000 layers.

The paper won the best paper award at CVPR 2016.

Design details that mattered

Bottleneck blocks. For the deepest models, each residual block used three layers: a 1×1 convolution to reduce dimensions, a 3×3 convolution, and a 1×1 convolution to restore dimensions. This kept computation affordable.

Batch normalization after convolutions helped training stability.

Projection shortcuts. When a block changes the size of its output, a simple linear projection on the shortcut makes the shapes match.

The legacy: everywhere

ResNet’s influence extends far beyond image classification.

  • Transformers wrap every attention layer and every feed-forward layer in residual connections. Without them, stacking dozens of transformer layers would be far harder.
  • Object detection and segmentation systems used ResNet backbones for years.
  • Diffusion models for image generation use residual blocks throughout.
  • ResNet-50 became a standard benchmark for measuring AI hardware performance.

Why the idea endures

ResNet’s lesson is about trainability. A network’s expressive power is useless if gradient descent cannot find good parameters. By giving signals and gradients a direct path through the network, skip connections turned depth from a liability into an asset — and made the very deep models of the following decade possible.

Frequently asked questions

What is a skip connection in simple terms?

It is a shortcut that carries a layer’s input around a group of layers and adds it to their output. The layers only have to learn a correction to the input instead of a completely new representation, and gradients can travel back through the shortcut during training without fading away.

Is ResNet still used today?

Yes. Plain ResNets are still common as fast, reliable image backbones, as reference models in research and as a standard workload for measuring AI hardware. More importantly, residual connections are built into almost every modern architecture, including transformers and diffusion models.

What does “152 layers” mean for computation?

Depth alone does not decide cost. The ResNet authors used bottleneck blocks that shrink and re-expand the number of channels, so their 152-layer network needed less computation than some shallower networks of the time, such as the VGG models, while being far more accurate.

Why did deeper networks perform worse before ResNet?

Because they were harder to optimize, not because they overfit. Training error itself went up as layers were added, which showed that gradient descent struggled to find good parameters in very deep plain networks. Residual connections made those parameters much easier to reach.