Architecture
2 milestones in AI history
ResearchDeep Learning Breakthrough
ResNet: Deeper Than Ever
Microsoft Research introduced ResNet with skip connections (residual connections), enabling the training of networks with 152+ layers — 8x deeper than previous networks. ResNet won ImageNet 2015 with 3.57% error, surpassing human-level performance (5.1%) for the first time.
Kaiming HeXiangyu ZhangMicrosoft Research
ResearchDeep Learning Breakthrough
Attention Is All You Need: The Transformer
Eight researchers at Google published 'Attention Is All You Need,' introducing the Transformer architecture. It replaced recurrence with self-attention mechanisms that could process entire sequences in parallel. The paper's title was deliberately bold — and proved prescient.
Ashish VaswaniNoam ShazeerGoogle BrainGoogle Research