Thanks to visit codestin.com
Credit goes to github.com

Skip to content

test failure in LLE #575

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
mblondel opened this issue Jan 20, 2012 · 16 comments
Closed

test failure in LLE #575

mblondel opened this issue Jan 20, 2012 · 16 comments

Comments

@mblondel
Copy link
Member

I'm (still?) getting a test failure in the LLE module.

======================================================================
FAIL: test_locally_linear.test_lle_simple_grid
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/pymodules/python2.6/nose/case.py", line 183, in runTest
    self.test(*self.arg)
  File "/home/mathieu/Desktop/projects/scikit-learn/sklearn/manifold/tests/test_locally_linear.py", line 67, in te
st_lle_simple_grid
    assert_lower(np.linalg.norm(X_reembedded - clf.embedding_), tol)
  File "/home/mathieu/Desktop/projects/scikit-learn/sklearn/manifold/tests/test_locally_linear.py", line 15, in as
sert_lower
    assert a < b, message
AssertionError: 0.67260012006758951 is not lower than 0.10000000000000001
>>  assert 0.67260012006758951 < 0.10000000000000001, '0.67260012006758951 is not lower than 0.10000000000000001'

The strange thing is that the test passes when test_locally_linear.py is run individually but fails when run from make test-code.

BTW, make test is supposed to run the test-code and test-doc targets but I noticed that when there is a failure in test-code, test-doc is not executed. This means that a failure in a unrelated module to your changes can hide doctest failures.

If the test is difficult to fix, we could just skip it, maybe?
@ogrisel
Copy link
Member

ogrisel commented Jan 20, 2012

Is there some use of a np.random global random state rather than using an explicitly seeded rng?

@mblondel
Copy link
Member Author

@ogrisel: it seems that the tests don't use a random state. I'd rather skip the test if it's fragile. What do you think, @jakevdp?

EDIT: I meant to write "the tests don't use a global random state".

@ogrisel
Copy link
Member

ogrisel commented Jan 25, 2012

The test is using explicit seeding. Maybe the randomness comes from the arpack solver? Or it can be a consequence on the other memory bug in the ball tree implementation.

@jakevdp
Copy link
Member

jakevdp commented Jan 25, 2012

There is some randomness in ARPACK, I believe. When calling the fortran routines, one can specify a starting vector - I think that's selected randomly if not supplied.

@ogrisel
Copy link
Member

ogrisel commented Jan 25, 2012

I think we should supply a random init vector based on the random_state of the wrapping scikit-learn class to make that behavior controllable.

@mblondel
Copy link
Member Author

+1 with @ogrisel's solution

@jakevdp
Copy link
Member

jakevdp commented Jan 25, 2012

+1 The starting vector can be specified with the v0 keyword to eigsh.

@jakevdp jakevdp mentioned this issue Jan 26, 2012
Merged
@amueller
Copy link
Member

Should this be closed?

@jakevdp
Copy link
Member

jakevdp commented Apr 21, 2012

I think this test can still fail (not sure why). The random_state PR was just a way to get more control in the testing. @mblondel, do you still see this failure?

@mblondel
Copy link
Member Author

Yes, I still see the failure...

@amueller
Copy link
Member

This issue only shows with scipy 0.9 as far as I can tell.

@amueller
Copy link
Member

amueller commented May 6, 2012

Should be fixed in da022b8. Can you confirm please?

@andrewmilkowski
Copy link

Reporting it here (getting similar error during while running nosetests sklearn ) using latest git master (with the patch, verified the change in the source code, but otherwise ready to run examples and learn! thanks for the project

Environment:

[amilkowski@localhost scikit-learn]$ uname -a
Linux localhost.localdomain 3.4.3-1.fc17.x86_64 #1 SMP Mon Jun 18 19:53:17 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

FAIL: test_locally_linear.test_lle_simple_grid

Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/opt/git/python/scikit-learn/sklearn/manifold/tests/test_locally_linear.py", line 64, in test_lle_simple_grid
assert_less(np.linalg.norm(X_reembedded - clf.embedding_), tol)
AssertionError: 0.51937584624682986 not less than 0.1
"""Fail immediately, with the given message."""

raise self.failureException('0.51937584624682986 not less than 0.1')

also dumping scipy version if it helps

[amilkowski@localhost scikit-learn]$ sudo yum list scipy

Installed Packages
scipy.x86_64 0.10.1-1.fc17 @fedora

@amueller
Copy link
Member

This is hopefully fixed in #1039

@amueller
Copy link
Member

Fixed in #1039. Thanks @jakevdp

@mblondel
Copy link
Member Author

@amueller You have a good memory :) I was searching for this ticket to close it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@ogrisel @mblondel @andrewmilkowski @amueller @jakevdp