Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Wrong normalization for pseudolikelihood in Restricted Boltzmann Machine. #23179

Open
@KVasya

Description

@KVasya

Describe the bug

Pseudolikelihood of a sample under Restricted Boltzmann machine includes the number of features as a multiplier.
Thus pseudolikelihood reflects the dimensionality of visible units, apart from the sample probability.
I believe this multiplier should be removed.

Steps/Code to Reproduce

from sklearn.neural_network import BernoulliRBM
import numpy as np

RBM_= BernoulliRBM(verbose=1)
X_10= np.ones([10,10])
X_1000= np.ones([10,1000])

print('Learning RBM with 10 visible units:')
RBM_.fit(X_small)
print(100*'')
print('Learning RBM with 1000 visible units')
RBM
.fit(X_large)

Expected Results

The pseudolikelihoods for 10 and 1000 visible units should be of the same order at corresponding learning stages.

Actual Results

Learning RBM with 10 visible units:
[BernoulliRBM] Iteration 1, pseudo-likelihood = -0.19, time = 0.00s
[BernoulliRBM] Iteration 2, pseudo-likelihood = -0.28, time = 0.00s
[BernoulliRBM] Iteration 3, pseudo-likelihood = -0.22, time = 0.00s
[BernoulliRBM] Iteration 4, pseudo-likelihood = -0.12, time = 0.00s
[BernoulliRBM] Iteration 5, pseudo-likelihood = -0.10, time = 0.00s
[BernoulliRBM] Iteration 6, pseudo-likelihood = -0.07, time = 0.00s
[BernoulliRBM] Iteration 7, pseudo-likelihood = -0.07, time = 0.00s
[BernoulliRBM] Iteration 8, pseudo-likelihood = -0.08, time = 0.00s
[BernoulliRBM] Iteration 9, pseudo-likelihood = -0.10, time = 0.01s
[BernoulliRBM] Iteration 10, pseudo-likelihood = -0.06, time = 0.00s


Learning RBM with 1000 visible units
[BernoulliRBM] Iteration 1, pseudo-likelihood = -3.80, time = 0.01s
[BernoulliRBM] Iteration 2, pseudo-likelihood = -0.76, time = 0.02s
[BernoulliRBM] Iteration 3, pseudo-likelihood = -0.94, time = 0.01s
[BernoulliRBM] Iteration 4, pseudo-likelihood = -1.97, time = 0.01s
[BernoulliRBM] Iteration 5, pseudo-likelihood = -2.01, time = 0.01s
[BernoulliRBM] Iteration 6, pseudo-likelihood = -0.61, time = 0.01s
[BernoulliRBM] Iteration 7, pseudo-likelihood = -1.06, time = 0.01s
[BernoulliRBM] Iteration 8, pseudo-likelihood = -1.09, time = 0.01s
[BernoulliRBM] Iteration 9, pseudo-likelihood = -0.28, time = 0.01s
[BernoulliRBM] Iteration 10, pseudo-likelihood = -1.01, time = 0.01s

Versions

System:
    python: 3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC 7.3.0]
executable: /home/x/PycharmProjects/RBM/env/bin/python
   machine: Linux-4.15.0-142-generic-x86_64-with-debian-stretch-sid

Python dependencies:
          pip: 22.0.4
   setuptools: 40.8.0
      sklearn: 1.0.2
        numpy: 1.21.5
        scipy: 1.7.3
       Cython: None
       pandas: None
   matplotlib: 3.5.1
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions