[MRG] add warning when importing old or new pickle. #7248

amueller · 2016-08-25T20:31:18Z

Adds a warning when loading a possibly incompatible pickle from a different version of sklearn.
Fixes #7135.

amueller · 2016-08-25T21:00:59Z

So, this works nicely. My test is a bit crazy, removing __getstate__ from BaseEstimator by monkey-patching, to emulate the old (current) behavior. I could also overwrite the __getstate__ on a single estimator, making it the vanilla return self.__dict__.

ping @jnothman ;)

amueller · 2016-08-25T21:01:56Z

one question remains, should we leave the __version__ string stored in the estimator in unpacking? We can remove it to make pickle.loads(pickle.dumps(est)) a no-op.

jnothman · 2016-08-26T01:56:32Z

sklearn/base.py

+        pickle_version = state.get("__version__", "pre-0.18")
+        if pickle_version != __version__:
+            warnings.warn(
+                "Trying to unpickle estimator from version {} when with using "


drop "with"

Given the lack of traceback with warnings, we should probably mention the estimator. (Although we could use stacklevel, the previous stack entries will be in the pickle module.) The annoyance of that will be that the message will be printed for every estimator loaded.

amueller · 2016-08-26T16:09:41Z

done

lesteve · 2016-08-27T07:49:20Z

sklearn/tests/test_base.py

+
+    # check that no warning is raised for external estimators
+    DecisionTreeClassifier.__module__ = "notsklearn"
+    assert_no_warnings(pickle.loads, tree_pickle_noversion)


You probably need to reset the __module__ after the assert_no_warnings. I am guessing this is the reason of the test failures.

oh yeah that's true.

jnothman · 2016-08-27T11:37:28Z

Is there a way to make this monkey-patching not break stuff if tests are run with parallelism?

amueller · 2016-08-29T16:06:44Z

maybe I should just remove the monkey patching. It's evil.

amueller · 2016-08-29T16:10:11Z

Ah, wait, it's the fiddeling with the __module__ that breaks tests, not the way more intrusive monkey-patching of BaseEstimator... I'll fix the module stuff in a bit, not sure if the monkey patching of the BaseEstimator test is worth the potential trouble.

GaelVaroquaux · 2016-08-29T16:15:07Z

sklearn/base.py

+        if type(self).__module__.startswith('sklearn.'):
+            return dict(self.__dict__.items(), __version__=__version__)
+        else:
+            return dict(self.__dict__.items())


Why don't we simply add a version_ attribute to BaseEstimator?

That way the version number of sklearn is stored on pickle without such a hack for everything that inherits from BaseEstimator.

Why hack? This adds the information to the file when the estimator is stored.
What would be the benefit of a version_ attribute outside of serialization?

The __setstate__ would still need to be changed the same way, right? (only without a pop)

Why hack? This adds the information to the file when the estimator is stored.
What would be the benefit of a version_ attribute outside of serialization?

Clarity and simplicity. Here we are relying on overriding the pickling
mechanism. We wouldn't need so. Less overriding of __ methods leads to
less tricky bug to debug.

The setstate would still need to be changed the same way, right? (only
without a pop)

I believe so.

I'm not fundamentally against adding it as an attribute to BaseEstimator but I'd rather make the attribute private to not pollute the autocomplete too much.

I'm not fundamentally against adding it as an attribute to BaseEstimator but I'd rather make the attribute private to not pollute the autocomplete too much.

+1

I think the class attribute issue is the bigger problem here. It doesn't get pickled unless it's set on the instance, does it?

Can we still have that only on models inside scikit-learn? It would be a class
attribute,

+1: see in the class of BaseEstimator. It should be very simple, just
adding

_version_ = sklearn.__version__

before the init in the class.

The only doubt that I have is about circular imports by accessing the
central sklearn module.

I think the class attribute issue is the bigger problem here. It doesn't get pickled unless it's set on the instance, does it?

Hum, you might be right. Darn, that may break my plan of simplicity. Overriding a __ method always worries me, so I'd like to try to avoid, if possible (the reason is that it gives non-standard behavior to objects, hence more difficult comprehension of a non library, and more difficult debuging.

@GaelVaroquaux does that mean you don't object to the current solution?

ping @GaelVaroquaux [sorry, I know you're busy]

amueller · 2016-09-07T22:04:59Z

@jnothman wdyt?

jnothman · 2016-09-08T00:06:47Z

IsotonicRegression overrides __[gs]etstate__ without calling super.
This should be (part of) a common test. (Sorry for the added work.)

lesteve · 2016-09-08T13:09:53Z

sklearn/tests/test_base.py

+
+    # check that not including any version also works:
+    # TreeNoVersion has no getstate, like pre-0.18
+    tree = TreeNoVersion().fit(iris.data, iris.target)


This is pretty neat compared to the previous version with monkey-patching!

add tests for pickle warning add test that loading something stored with no custom __getstate__ will still work (and raise a warning) added "within sklearn" check, included estimator name in warning message. add that pickle warnings only apply to sklearn estimators. changed tests to be threadsave and contain no monkey business

amueller · 2016-09-08T14:40:27Z

@jnothman added the tests to common tests. Only isotonic is not tested (because 1d input)....

jnothman · 2016-09-08T14:58:40Z

sklearn/utils/estimator_checks.py


    # pickle and unpickle!
    pickled_estimator = pickle.dumps(estimator)
+    assert_true(b"version" in pickled_estimator)


Hmm... Perhaps only if module.startswith('sklearn.')!

jnothman · 2016-09-08T15:21:06Z

sklearn/utils/estimator_checks.py

    # pickle and unpickle!
    pickled_estimator = pickle.dumps(estimator)
+    if Estimator.__module__.startswith('sklearn.'):
+        assert_true(b"version" in pickled_estimator)


"version", no?

took me a while to figure out what you mean ;) __version__? the underscores get mangled, I think. I felt this was enough. I can try to get the mangling to work, but I'd rather merge this and get 0.18 out of the door...

ogrisel · 2016-09-08T19:23:45Z

Apart from https://github.com/scikit-learn/scikit-learn/pull/7248/files#r78072506, I am also +1.

ogrisel · 2016-09-10T12:17:35Z

Merging!

amueller · 2016-09-12T17:04:20Z

sweet, thanks :)

amueller · 2016-09-12T17:04:34Z

needs a whatsnew, right?

…cikit-learn#7400)

amueller mentioned this pull request Aug 25, 2016

Model persistence warnings across versions #7135

Closed

amueller changed the title ~~add warning when importing old or new pickle.~~ [MRG] add warning when importing old or new pickle. Aug 25, 2016

amueller mentioned this pull request Aug 25, 2016

StandardScaler's bug when using pickle #6596

Closed

MechCoder added this to the 0.18 milestone Aug 26, 2016

jnothman reviewed Aug 26, 2016
View reviewed changes

lesteve reviewed Aug 27, 2016
View reviewed changes

GaelVaroquaux reviewed Aug 29, 2016
View reviewed changes

lesteve reviewed Sep 8, 2016
View reviewed changes

amueller force-pushed the pickle_warning branch from 1edd87c to b28cfbc Compare September 8, 2016 14:41

make IsotonicRegression __setstate__ and __getstate__ call super

c980d98

jnothman reviewed Sep 8, 2016
View reviewed changes

conditional test for version in common tests

106c397

jnothman reviewed Sep 8, 2016
View reviewed changes

don't use __version__, use _sklearn_version

461ae84

ogrisel merged commit b4872fe into scikit-learn:master Sep 10, 2016

amueller added a commit to amueller/scikit-learn that referenced this pull request Sep 12, 2016

add whatsnew for scikit-learn#7248

6449fb3

jnothman pushed a commit that referenced this pull request Sep 13, 2016

[MRG] add whatsnew for #7248 (unpickle version warning) (#7400)

4d9a12d

rsmith54 pushed a commit to rsmith54/scikit-learn that referenced this pull request Sep 14, 2016

[MRG] add warning when importing old or new pickle. (scikit-learn#7248)

cfae9aa

rsmith54 pushed a commit to rsmith54/scikit-learn that referenced this pull request Sep 14, 2016

[MRG] add whatsnew for scikit-learn#7248 (unpickle version warning) (s…

074796e

…cikit-learn#7400)

TomDLT pushed a commit to TomDLT/scikit-learn that referenced this pull request Oct 3, 2016

[MRG] add warning when importing old or new pickle. (scikit-learn#7248)

c62b049

TomDLT pushed a commit to TomDLT/scikit-learn that referenced this pull request Oct 3, 2016

[MRG] add whatsnew for scikit-learn#7248 (unpickle version warning) (s…

45f416d

…cikit-learn#7400)

Sundrique pushed a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017

[MRG] add warning when importing old or new pickle. (scikit-learn#7248)

05bdd7d

Sundrique pushed a commit to Sundrique/scikit-learn that referenced this pull request Jun 14, 2017

[MRG] add whatsnew for scikit-learn#7248 (unpickle version warning) (s…

f52d19a

…cikit-learn#7400)

paulha pushed a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017

[MRG] add warning when importing old or new pickle. (scikit-learn#7248)

22ef02e

paulha pushed a commit to paulha/scikit-learn that referenced this pull request Aug 19, 2017

[MRG] add whatsnew for scikit-learn#7248 (unpickle version warning) (s…

b35f363

…cikit-learn#7400)

tirkarthi mentioned this pull request Jun 4, 2018

Scikit-learn warning mailgun/talon#158

Open

rth mentioned this pull request Jun 30, 2021

Give a more informed warning message when unpickling a estimator with a different sklearn version #20306

Closed

arubiales mentioned this pull request Jun 30, 2021

[MRG] BaseEstimator added url with info about problems to load with pickle models from differents sklearn versions #20434

Merged

Uh oh!

[MRG] add warning when importing old or new pickle. #7248

[MRG] add warning when importing old or new pickle. #7248

Uh oh!

Conversation

amueller commented Aug 25, 2016

Uh oh!

amueller commented Aug 25, 2016

Uh oh!

amueller commented Aug 25, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amueller commented Aug 26, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jnothman commented Aug 27, 2016

Uh oh!

amueller commented Aug 29, 2016

Uh oh!

amueller commented Aug 29, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GaelVaroquaux Aug 29, 2016 via email

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

GaelVaroquaux Aug 30, 2016 via email

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amueller commented Sep 7, 2016

Uh oh!

jnothman commented Sep 8, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

amueller commented Sep 8, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ogrisel commented Sep 8, 2016

Uh oh!

ogrisel commented Sep 10, 2016

Uh oh!

amueller commented Sep 12, 2016

Uh oh!

amueller commented Sep 12, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone