Residual network skip connection block diagram

ResNet: Deeper Than Ever

Research BreakthroughsDeep Learning Breakthrough
Impact

What Happened

Microsoft Research introduced ResNet with skip connections (residual connections), enabling the training of networks with 152+ layers — 8x deeper than previous networks. ResNet won ImageNet 2015 with 3.57% error, surpassing human-level performance (5.1%) for the first time.

Why It Mattered

First AI system to beat humans at large-scale image recognition. Residual connections became a fundamental building block in virtually all deep architectures, including Transformers.

Key People

Organizations

Tags

Part of the Deep Learning Breakthrough (2012–2017) era · Browse all research breakthroughs · View all 2015 milestones

Related Milestones

AlexNet deep neural network architecture diagram
Research

AlexNet: The ImageNet Moment

AlexNet, a deep convolutional neural network, won the ImageNet competition by a staggering margin — reducing the error rate from 26% to 16%. Trained on two NVIDIA GTX 580 GPUs, it was dramatically deeper and more powerful than previous entries. The AI community was stunned.

Alex KrizhevskyIlya SutskeverUniversity of Toronto
Google DeepMind logo
Research

DeepMind's DQN Masters Atari Games

DeepMind demonstrated a deep reinforcement learning agent (Deep Q-Network) that learned to play Atari 2600 games directly from pixel inputs, achieving superhuman performance on many games with no task-specific engineering. Google acquired DeepMind for ~$500 million shortly after.

Volodymyr MnihDemis HassabisDeepMind
The Transformer model architecture diagram from Attention Is All You Need
Research

Attention Is All You Need: The Transformer

Eight researchers at Google published 'Attention Is All You Need,' introducing the Transformer architecture. It replaced recurrence with self-attention mechanisms that could process entire sequences in parallel. The paper's title was deliberately bold — and proved prescient.

Ashish VaswaniNoam ShazeerGoogle BrainGoogle Research
LeNet-5 convolutional neural network architecture
Research

LeNet: Convolutional Neural Networks

Yann LeCun demonstrated that convolutional neural networks (CNNs) could be trained with backpropagation to recognize handwritten digits. The refined LeNet-5 (1998) achieved 99%+ accuracy on MNIST and was deployed by banks to read checks — running in ATMs for years.

Yann LeCunAT&T Bell Labs
Tomáš Mikolov, lead author of Word2Vec
Research

Word2Vec: Words as Vectors

Google researchers published Word2Vec, showing that relatively small neural networks could efficiently learn meaningful vector representations of words from large text corpora. The famous example `king - man + woman ≈ queen` made the idea vivid: semantic relationships could be captured geometrically in vector space.

Tomas MikolovGoogle

Get the latest AI milestones as they happen

Join the newsletter. No spam, just signal.