Description
Describe the bug
Pseudolikelihood of a sample under Restricted Boltzmann machine includes the number of features as a multiplier.
Thus pseudolikelihood reflects the dimensionality of visible units, apart from the sample probability.
I believe this multiplier should be removed.
Steps/Code to Reproduce
from sklearn.neural_network import BernoulliRBM
import numpy as np
RBM_= BernoulliRBM(verbose=1)
X_10= np.ones([10,10])
X_1000= np.ones([10,1000])
print('Learning RBM with 10 visible units:')
RBM_.fit(X_small)
print(100*'')
print('Learning RBM with 1000 visible units')
RBM.fit(X_large)
Expected Results
The pseudolikelihoods for 10 and 1000 visible units should be of the same order at corresponding learning stages.
Actual Results
Learning RBM with 10 visible units:
[BernoulliRBM] Iteration 1, pseudo-likelihood = -0.19, time = 0.00s
[BernoulliRBM] Iteration 2, pseudo-likelihood = -0.28, time = 0.00s
[BernoulliRBM] Iteration 3, pseudo-likelihood = -0.22, time = 0.00s
[BernoulliRBM] Iteration 4, pseudo-likelihood = -0.12, time = 0.00s
[BernoulliRBM] Iteration 5, pseudo-likelihood = -0.10, time = 0.00s
[BernoulliRBM] Iteration 6, pseudo-likelihood = -0.07, time = 0.00s
[BernoulliRBM] Iteration 7, pseudo-likelihood = -0.07, time = 0.00s
[BernoulliRBM] Iteration 8, pseudo-likelihood = -0.08, time = 0.00s
[BernoulliRBM] Iteration 9, pseudo-likelihood = -0.10, time = 0.01s
[BernoulliRBM] Iteration 10, pseudo-likelihood = -0.06, time = 0.00s
Learning RBM with 1000 visible units
[BernoulliRBM] Iteration 1, pseudo-likelihood = -3.80, time = 0.01s
[BernoulliRBM] Iteration 2, pseudo-likelihood = -0.76, time = 0.02s
[BernoulliRBM] Iteration 3, pseudo-likelihood = -0.94, time = 0.01s
[BernoulliRBM] Iteration 4, pseudo-likelihood = -1.97, time = 0.01s
[BernoulliRBM] Iteration 5, pseudo-likelihood = -2.01, time = 0.01s
[BernoulliRBM] Iteration 6, pseudo-likelihood = -0.61, time = 0.01s
[BernoulliRBM] Iteration 7, pseudo-likelihood = -1.06, time = 0.01s
[BernoulliRBM] Iteration 8, pseudo-likelihood = -1.09, time = 0.01s
[BernoulliRBM] Iteration 9, pseudo-likelihood = -0.28, time = 0.01s
[BernoulliRBM] Iteration 10, pseudo-likelihood = -1.01, time = 0.01s
Versions
System:
python: 3.7.3 (default, Mar 27 2019, 22:11:17) [GCC 7.3.0]
executable: /home/x/PycharmProjects/RBM/env/bin/python
machine: Linux-4.15.0-142-generic-x86_64-with-debian-stretch-sid
Python dependencies:
pip: 22.0.4
setuptools: 40.8.0
sklearn: 1.0.2
numpy: 1.21.5
scipy: 1.7.3
Cython: None
pandas: None
matplotlib: 3.5.1
joblib: 1.1.0
threadpoolctl: 3.1.0
Built with OpenMP: True