Historical milestone · 2006
Deep belief nets
Reviewed through September 18, 2026
2006 · Historical milestone
Deep belief nets
- Era
- 2000s
- Theme
- Machine-learning foundations
- Evidence form
- Greedy layer-wise unsupervised training initialized deep networks before supervised fine-tuning
- School / paradigm
- Deep generative learning
- Institution / context
- University of Toronto / National University of Singapore
- Researchers
- Geoffrey Hinton; Simon Osindero; Yee-Whye Teh
Researcher index
Geoffrey Hinton
Deep representation learning · CMU / Toronto
Backpropagation and deep belief nets
Why it still matters. Central architect of the connectionist revival and deep learning.
Representative source for this researcher — not necessarily the source of this milestone: https://doi.org/10.1162/neco.2006.18.7.1527 (opens in a new tab)
School of thought
Connectionism
Matched on representative researcher.
Cognition emerges from learned distributed representations and weighted interactions among simple units.
Critique. Opacity, data/compute demands, unstable optimization, and weak guarantees or causal grounding.
Modern descendants. Foundation models, multimodal networks, representation learning, and differentiable agents.
Understand
Plain-language record, transferred from the reviewed source module.
Theory or experimental setup. Provided a practical training route for deep representations when end-to-end optimization was difficult.
Result / historical claim. Generative assumptions and pretraining were later displaced in many tasks by better optimization, activations, data, and compute.
Apply
Professional implication, only where the reviewed record states one.
The checked-in record does not state a separate professional application for this entry. The topic page places it in the wider research lineage: .
Verify
Evidence status, stated limitations, and the external sources this record actually carries.
Evidence form. Greedy layer-wise unsupervised training initialized deep networks before supervised fine-tuning
Limitation / debate. Layer-wise pretraining, generative representation learning, and the return of deep neural networks.
Source status. This milestone row does not carry a primary-source URL in the approved export, and we do not have a verified link for it in our own research. We do not guess one.
No primary-source URL is recorded for this entry in our reviewed data. Rather than manufacture a citation, we link the Implement Agentic research page that carries the record.
Reproduce
A reproduction tutorial is linked only when one exists for this exact record.
A reproduction tutorial is not yet available for this entry. The closest reviewed material is .
Cite or share
APA-like: This historical record carries a year only, and no author or publisher of record in the checked-in data. An APA reference would have to invent that metadata.
BibTeX: BibTeX requires an author and publication venue. Historical lineage entries store a narrative record and its source link, not structured authorship, so the field would be fabricated.