Reinforcement learning agent-environment interaction diagram

TD-Gammon: Reinforcement Learning Plays Backgammon

Research BreakthroughsSecond AI Winter
Impact

What Happened

Gerald Tesauro created TD-Gammon, a neural network that learned to play backgammon at expert level through self-play using temporal difference reinforcement learning. It discovered novel strategies that surprised human experts.

Why It Mattered

Pioneering demonstration of reinforcement learning + neural networks. Foreshadowed AlphaGo's self-play approach by 24 years.

Key People

Organizations

Tags

Part of the Second AI Winter (1988–1993) era · Browse all research breakthroughs · View all 1992 milestones

Related Milestones

Research

Samuel's Checkers Program

Arthur Samuel created a checkers-playing program at IBM that could learn from experience, improving its play over time. He coined the term 'machine learning' to describe programs that learn without being explicitly programmed.

Arthur SamuelIBM
Google DeepMind logo
Research

DeepMind's DQN Masters Atari Games

DeepMind demonstrated a deep reinforcement learning agent (Deep Q-Network) that learned to play Atari 2600 games directly from pixel inputs, achieving superhuman performance on many games with no task-specific engineering. Google acquired DeepMind for ~$500 million shortly after.

Volodymyr MnihDemis HassabisDeepMind
NETtalk neural network back-propagation diagram
Research

NETtalk: Neural Network Learns to Speak

NETtalk was a neural network that learned to pronounce English text aloud, starting from babbling sounds and gradually becoming intelligible — mimicking how a child learns to speak. It captured public imagination and demonstrated backpropagation's potential.

Terrence SejnowskiCharles RosenbergJohns Hopkins University
LeNet-5 convolutional neural network architecture
Research

LeNet: Convolutional Neural Networks

Yann LeCun demonstrated that convolutional neural networks (CNNs) could be trained with backpropagation to recognize handwritten digits. The refined LeNet-5 (1998) achieved 99%+ accuracy on MNIST and was deployed by banks to read checks — running in ATMs for years.

Yann LeCunAT&T Bell Labs
LSTM recurrent neural network cell diagram
Research

Long Short-Term Memory (LSTM)

Hochreiter and Schmidhuber published the LSTM architecture, solving the vanishing gradient problem that plagued recurrent neural networks. LSTMs could learn long-range dependencies in sequential data by maintaining a memory cell with gates that controlled information flow.

Sepp HochreiterJürgen SchmidhuberTechnical University of Munich

Get the latest AI milestones as they happen

Join the newsletter. No spam, just signal.