Backpropagation Rediscovered

Research Breakthroughs Expert Systems Boom

Impact

At a glance

Date: 1986
Era: Expert Systems Boom (1980–1987)
Category: Research Breakthroughs
Impact: 5 / 5
Key people: David Rumelhart, Geoffrey Hinton, Ronald Williams
Organizations: UC San Diego, Carnegie Mellon University

What Happened

Rumelhart, Hinton, and Williams published 'Learning Representations by Back-propagating Errors' in Nature, demonstrating that backpropagation could train multi-layer neural networks effectively. The same year, the PDP (Parallel Distributed Processing) group published their influential two-volume work on connectionism.

Why It Mattered

Revived neural network research from its decade-long exile. Backpropagation became the standard training method for multi-layer neural networks and underpinned the deep learning wave that followed.

Key People

Organizations

Frequently asked questions

When did Backpropagation Rediscovered happen?+

Backpropagation Rediscovered took place in 1986.

Who was behind Backpropagation Rediscovered?+

For Backpropagation Rediscovered, key people included David Rumelhart, Geoffrey Hinton, and Ronald Williams and organizations involved were UC San Diego and Carnegie Mellon University.

Why was Backpropagation Rediscovered important?+

Revived neural network research from its decade-long exile. Backpropagation became the standard training method for multi-layer neural networks and underpinned the deep learning wave that followed.

Which era of AI history does Backpropagation Rediscovered belong to?+

Backpropagation Rediscovered is part of the Expert Systems Boom era (1980–1987) — a landmark, field-defining moment in the research breakthroughs category.

Related Milestones

1987Research

NETtalk: Neural Network Learns to Speak

NETtalk was a neural network that learned to pronounce English text aloud, starting from babbling sounds and gradually becoming intelligible — mimicking how a child learns to speak. It captured public imagination and demonstrated backpropagation's potential.

Terrence SejnowskiCharles RosenbergJohns Hopkins University

1982Research

Hopfield Networks: Physics Meets Neural Networks

Physicist John Hopfield showed that a type of recurrent neural network could serve as content-addressable memory, using concepts from statistical physics. The network would converge to stable states that could store and retrieve patterns — connecting neuroscience, physics, and computation.

John HopfieldCaltech

1974Research

Backpropagation Discovered (Initially Ignored)

Paul Werbos described the backpropagation algorithm in his PhD thesis — a method for training multi-layer neural networks by propagating errors backward through the network. However, in the anti-neural-network climate of the 1970s, the work went largely unnoticed.

Paul WerbosHarvard University

2006Research

Deep Belief Networks: Hinton Revives Deep Learning

Geoffrey Hinton published 'A Fast Learning Algorithm for Deep Belief Nets,' showing that deep neural networks could be effectively trained by pre-training each layer as a restricted Boltzmann machine. This solved the long-standing problem of training networks with many layers.

Geoffrey HintonSimon OsinderoUniversity of Toronto

1989Research

LeNet: Convolutional Neural Networks

Yann LeCun demonstrated that convolutional neural networks (CNNs) could be trained with backpropagation to recognize handwritten digits. The refined LeNet-5 (1998) achieved 99%+ accuracy on MNIST and was deployed by banks to read checks — running in ATMs for years.

Yann LeCunAT&T Bell Labs

Get the latest AI milestones as they happen

Join the newsletter. No spam, just signal.