Word2Vec: Words as Vectors

Research Breakthroughs Deep Learning Breakthrough

Impact

What Happened

Google researchers published Word2Vec, showing that relatively small neural networks could efficiently learn meaningful vector representations of words from large text corpora. The famous example `king - man + woman ≈ queen` made the idea vivid: semantic relationships could be captured geometrically in vector space.

Why It Mattered

Made word embeddings practical at scale and gave NLP a shared representation layer that was easy to train, reuse, and reason about. It helped bridge older statistical NLP and the neural language-model era that eventually led to transformers.

Key People

Tomas Mikolov

Organizations

Google

Related Milestones

2017.06Research

Attention Is All You Need: The Transformer

Eight researchers at Google published 'Attention Is All You Need,' introducing the Transformer architecture. It replaced recurrence with self-attention mechanisms that could process entire sequences in parallel. The paper's title was deliberately bold — and proved prescient.

Ashish VaswaniNoam ShazeerGoogle BrainGoogle Research

2012.09Research

AlexNet: The ImageNet Moment

AlexNet, a deep convolutional neural network, won the ImageNet competition by a staggering margin — reducing the error rate from 26% to 16%. Trained on two NVIDIA GTX 580 GPUs, it was dramatically deeper and more powerful than previous entries. The AI community was stunned.

Alex KrizhevskyIlya SutskeverUniversity of Toronto

2013.12Research

DeepMind's DQN Masters Atari Games

DeepMind demonstrated a deep reinforcement learning agent (Deep Q-Network) that learned to play Atari 2600 games directly from pixel inputs, achieving superhuman performance on many games with no task-specific engineering. Google acquired DeepMind for ~$500 million shortly after.

Volodymyr MnihDemis HassabisDeepMind

2014.06Research

Generative Adversarial Networks (GANs)

Ian Goodfellow introduced GANs — two neural networks (generator and discriminator) competing against each other, one creating fake data and the other trying to detect it. The concept allegedly came to him during a bar conversation. Yann LeCun called GANs 'the most interesting idea in the last 10 years in ML.'

Ian GoodfellowUniversité de Montréal

2015.12Research

ResNet: Deeper Than Ever

Microsoft Research introduced ResNet with skip connections (residual connections), enabling the training of networks with 152+ layers — 8x deeper than previous networks. ResNet won ImageNet 2015 with 3.57% error, surpassing human-level performance (5.1%) for the first time.

Kaiming HeXiangyu ZhangMicrosoft Research

Get the latest AI milestones as they happen

Join the newsletter. No spam, just signal.