Description
Description
ValueError: "array must not contain infs or NaNs" thrown when fitting an ExpSineSquared kernel to large dimension data
Steps/Code to Reproduce
import sklearn.gaussian_process as gp
from sklearn.gaussian_process.kernels import ExpSineSquared
while True:
xtr = np.random.rand(25,25)
ytr = np.random.rand(25)
model = gp.GaussianProcessRegressor(kernel=ExpSineSquared(),
alpha=1e-5,
n_restarts_optimizer=10,
normalize_y=True)
model.fit(xtr, ytr)Expected Results
No error is thrown
Actual Results
File "<ipython-input-39-119d71c23c9f>", line 1, in <module>
runfile('/Users/eric/Google Drive/Spatial conquest/Neural Network/GitHub Repository/tensorlayer/benchmarks/bug.py', wdir='/Users/eric/Google Drive/Spatial conquest/Neural Network/GitHub Repository/tensorlayer/benchmarks')
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 710, in runfile
execfile(filename, namespace)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/spyder/utils/site/sitecustomize.py", line 101, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "/Users/eric/Google Drive/Spatial conquest/Neural Network/GitHub Repository/tensorlayer/benchmarks/bug.py", line 19, in <module>
model.fit(xtr, ytr)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/sklearn/gaussian_process/gpr.py", line 232, in fit
bounds))
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/sklearn/gaussian_process/gpr.py", line 455, in _constrained_optimization
if convergence_dict["warnflag"] != 0:
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 193, in fmin_l_bfgs_b
**opts)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 328, in _minimize_lbfgsb
f, g = func_and_grad(x)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/optimize/lbfgsb.py", line 278, in func_and_grad
f = fun(x, *args)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 292, in function_wrapper
return function(*(wrapper_args + args))
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/optimize/optimize.py", line 63, in __call__
fg = self.fun(x, *args)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/sklearn/gaussian_process/gpr.py", line 209, in obj_func
theta, eval_gradient=True)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/sklearn/gaussian_process/gpr.py", line 419, in log_marginal_likelihood
except np.linalg.LinAlgError:
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/linalg/decomp_cholesky.py", line 81, in cholesky
check_finite=check_finite)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/scipy/linalg/decomp_cholesky.py", line 20, in _cholesky
a1 = asarray_chkfinite(a)
File "/Users/eric/Applications/anaconda/envs/NN_on_GPU/lib/python3.6/site-packages/numpy/lib/function_base.py", line 1215, in asarray_chkfinite
"array must not contain infs or NaNs")
ValueError: array must not contain infs or NaNs
Versions
Darwin-16.7.0-x86_64-i386-64bit
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
NumPy 1.13.1
SciPy 0.19.1
Scikit-Learn 0.19.0
Additional comment
This problem seems MacOS specific as it does not occur on my linux system with exactly the same versions
Linux-4.10.0-33-generic-x86_64-with-debian-stretch-sid
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:51:32)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)]
NumPy 1.13.1
SciPy 0.19.1
Scikit-Learn 0.19.0
In terms of investigation, the furthest i got was to see that kernel(self.X_train_) in sklearn/gaussian_process/gpr.py", line 419, log_marginal_likelihood was diverging to infinity