Helmholtz machine

From Wikipedia, the free encyclopedia
Jump to navigation Jump to search

The Helmholtz machine is a type of artificial neural network that can account for the hidden structure of a set of data by being trained to create a generative model of the original set of data.[1] The hope is that by learning economical representations of the data, the underlying structure of the generative model should reasonably approximate the hidden structure of the data set. A Helmholtz machine contains two networks, a bottom-up recognition network that takes the data as input and produces a distribution over hidden variables, and a top-down "generative" network that generates values of the hidden variables and the data itself.

Helmholtz machines are usually trained using an unsupervised learning algorithm, such as the wake-sleep algorithm.[2]. They are a precursor to variational autoencoders, which are instead trained using backpropagation. Helmholtz machines may also be used in applications requiring a supervised learning algorithm (e.g. character recognition, or position-invariant recognition of an object within a field).

See also[edit]

References[edit]

  1. ^ Peter, Dayan; Hinton, Geoffrey E.; Neal, Radford M.; Zemel, Richard S. (1995). "The helmholtz machine". Neural computation. 7: 889–904. closed access
  2. ^ Hinton, Geoffrey E.; Dayan, Peter; Frey, Brendan J.; Neal, Radford (1995-05-26). "The wake-sleep algorithm for unsupervised neural networks". Science. 268 (5214): 1158–1161. doi:10.1126/science.7761831. PMID 7761831. closed access

External links[edit]