Neurocomputing
Restricted Boltzmann Machines
Professur für Künstliche Intelligenz - Fakultät für Informatik
Restricted Boltzmann Machines
- Auto-encoders are not the only feature extractors that can be stacked.
Restricted Boltzmann Machines (RBM) are generative stochastic artificial neural networks that can learn a probability distribution of their inputs.
Their neurons form a bipartite graph with two groups of reciprocally connected units:
the visible units v (the inputs)
the hidden units h (the features or latent space).
Connections are bidirectional between v and h, but the neurons inside the two groups are independent from each other (restricted).
The goal of learning is to find the weights allowing the network to explain best the input data.
Hinton and Salakhutdinov (2006). Reducing the Dimensionality of Data with Neural Networks. Science 28, 504-507.
Restricted Boltzmann Machines
RBMs are a form of autoencoder where the input → feature weight matrix is the same as the feature → output matrix.
There are two steps:
The forward pass P(h∣x) propagates the visible units activation to the hidden units.
The backward pass P(x∣h) reconstructs the visible units from the the hidden units.
If the weight matrix is correctly chosen, the reconstructed input should “match” the original input: the data is explained.
Restricted Boltzmann Machines
- The visible and units are generally binary units (0 or 1), with a probability defined by the weights and biases and the logistic function:
P(hj=1∣v)=σ(i∑Wijvi+cj)
P(vi=1∣h)=σ(i∑Wjihj+bi)
- The weight matrix W and the biases b,c are the parameters θ of a probability distribution over the activation of the visible and hidden units.
The goal is to find the parameters which explain best the data (visible units), i.e. the ones maximizing the log-likelihood of the model for the data (v1,…,vN).
We use maximum likelihood estimation (MLE) to maximize the log-likelihood of the model:
θmaxL(θ)=Ev∼D[logPθ(v)]
Hinton and Salakhutdinov (2006). Reducing the Dimensionality of Data with Neural Networks. Science 28, 504-507.
Restricted Boltzmann Machines
- In practice, MLE is not tractable in a RBM, as we cannot estimate the joint probability P(v,h) of v and h (too many combinations are possible).
P(v)=h∑P(v,h)
- The main trick in energy-based models is to rewrite the probabilities using an energy function E(v,h):
P(v)=h∑P(v,h)=∑v,hexp−E(v,h)∑hexp−E(v,h)=Z1h∑exp−E(v,h)
where:
Z=v,h∑exp−E(v,h)=v∑P(v)h∑exp−E(v,h)
is the partition function (a normalizing term).
- The probabilities come from a Gibbs distribution (or Boltzmann distribution) parameterized by the energy of the system. This is equivalent to a simple softmax over the energy…
Hinton and Salakhutdinov (2006). Reducing the Dimensionality of Data with Neural Networks. Science 28, 504-507.
Restricted Boltzmann Machines
- Having reformulated the probabilities in terms of energy:
P(v)=Z1h∑exp−E(v,h)
we can introduce the free energy of the model for a sample v (how surprising is the input v for the model):
F(v)=−logh∑exp−E(v,h)
- The log-likelihood of the model for a sample v of the training data (v1,…,vN) becomes:
logP(v)=logZ1h∑exp−E(v,h)=−F(v)+logZ=−F(v)+v∑P(v)F(v)
Note that the second term sums over all possible inputs v.
Maximizing the log-likelihood of the model on the training data can be done using gradient ascent by following this gradient:
∇θL(θ)=Ev[∇θlogP(vi)]=Ev[−∇θF(v)+v∑P(v)∇θF(v)]
Restricted Boltzmann Machines
- The free energy for a RBM with binary neurons is fortunately known analytically:
F(v)=−i∑bivi−j∑log(1+exp∑iWijvi+cj)
so finding the gradient w.r.t θ=(W,b,c) of the first term on the r.h.s (the free energy of the sample) is easy:
∇θlogP(v)=−∇θF(v)+v∑P(v)∇θF(v)
- In particular, the gradient w.r.t the matrix W is the outer product between v and P(h∣v):
∇WF(v)=−v×P(h∣v)
The problem is the second term: we would need to integrate over all possible values of the inputs v, what is not tractable.
We will therefore make an approximation using Gibbs sampling (a variant of Monte-Carlo Markov Chain sampling - MCMC) to estimate that second term.
Gibbs sampling
Gibbs sampling consists of repeatedly applying the encoder P(h∣v) and the decoder P(v∣h) on the input.
We start by setting v0=v using a training sample.
We obtain h0 by computing P(h∣v0) and sampling it.
We obtain v1 by computing P(v∣h0) and sampling it.
…
We obtain vk by computing P(v∣hk−1) and sampling it.
After enough iterations k, we should have a good estimate of P(v,h).
The k iterations have generated enough reconstructions of v to cover the distribution of v.
Contrastive divergence
We set v0=v on a training sample and let Gibbs sampling iterate for k iterations until we obtain vk=v∗.
Contrastive divergence (CD-k) shows that the gradient of the log-likelihood can be approximated by:
∇WlogP(v)=−∇WF(v)+v∑P(v)∇WF(v)≈v×P(h∣v)−v∗×P(h∣v∗)
The gradient of the log-likelihood is the difference between the initial explanation of v by the model, and its explanation after k iterations (relaxation).
If the model is good, the reconstruction v∗ is the same as the input v, so the gradient is zero.
An input v is likely under the RBM model if it is able to reconstruct it, i.e. when it is not surprising (the free energy is low).
In practice, k=1 gives surprisingly good results, but RBMs are very painful to train (hyperparameters)…
Hinton GE, Osindero S, Teh YW (2006). “A fast learning algorithm for deep belief nets”. Neural Computation. 18 (7): 1527–54. doi:10.1162/neco.2006.18.7.1527.
Deep Belief Networks = stacked RBMs
A Deep Belief Network (DBM) is a simple stack of RBMS, trained using greedy layer-wise learning.
The “bottom” parts of the DBM become unidirectional when learning the top part.
Hinton GE, Osindero S, Teh YW (2006). “A fast learning algorithm for deep belief nets”. Neural Computation. 18 (7): 1527–54. doi:10.1162/neco.2006.18.7.1527.
Application: Finding cats on the internet
Andrew Ng and colleagues (Google, Stanford) used a similar technique to train a deep belief network on color images (200x200) taken from 10 million random unlabeled Youtube videos.
Each layer was trained greedily. They used a particular form of autoencoder called restricted Boltzmann machines (RBM) and a couple of other tricks (receptive fields, contrast normalization).
Training was distributed over 1000 machines (16.000 cores) and lasted for three days.
There was absolutely no task: the network just had to watch youtube videos.
After learning, they visualized what the neurons had learned.
Quoc Le et al. (2013). Building High-level Features Using Large Scale Unsupervised Learning. ICASSP. http://ieeexplore.ieee.org/document/6639343
Application: Finding cats on the internet
After training, some neurons had learned to respond uniquely to faces, or to cats, without ever having been instructed to.
The network can then be fine-tuned for classification tasks, improving the pre-AlexNet state-of-the-art on ImageNet by 70%.
Quoc Le et al. (2013). Building High-level Features Using Large Scale Unsupervised Learning. ICASSP. http://ieeexplore.ieee.org/document/6639343
