WARNING: this is not my main code, and there is no warranty attached!
= Generative Stochastic Network =
- A simple implementation of GSN according to (Bengio et al., 2013)
= Convolutional Neural Network =
- A naive implementation (purely using Matlab)
- Pooling: max (Jonathan Masci's code) and average
- Not for serious use!
= Restricted Boltzmann Machine & Deep Belief Networks =
- Binary/Gaussian Visible Units + Binary Hidden Units
- Enhanced Gradient, Adaptive Learning Rate
- Adadelta for RBM
- Contrastive Divergence
- (Fast) Persistent Contrastive Divergence
- Parallel Tempering
- DBN: Up-down Learning Algorithm
= Deep Boltzmann Machine =
- Binary/Gaussian Visible Units + Binary Hidden Units
- (Persistent) Contrastive Divergence
- Enhanced Gradient, Adaptive Learning Rate
- Two-stage Pretraining Algorithm (example)
- Centering Trick (fixed center variables only)
= Denoising Autoencoder (Tied Weights) =
- Binary/Gaussian Visible Units + Binary(Sigmoid)/Gaussian Hidden Units
- tanh/sigm/relu nonlinearities
- Shallow: sparsity, contractive, soft-sparsity (log-cosh) regularization
- Deep: stochastic backprop
- Adagrad, Adadelta
= Multi-layer Perceptron =
- Stochastic Backpropagation, Dropout
- tanh/sigm/relu nonlinearities
- Adagrad, Adadelta
- Balanced minibatches using crossvalind()