FIX : first pass at fixing nbagg close issue #4456

tacaswell · 2015-05-22T15:34:17Z

Add callback to destroy figure when the user hits the red x
in the notebook.

This is not the correct solution, but it (might) work for now.

closes #4281 ?

efiring · 2015-05-22T17:47:48Z

Why do you say this is not the correct solution?

tacaswell · 2015-05-22T18:34:21Z

The registration of the Gcf.destroy method needs to only be done if the user is using pyplot.

On the other hand, this code path may be only accessed through pyplot so it is ok (and this relates back to my, @OceanWolf , and @fariza long running attempt to pull pyplot related code out of the backends).

tacaswell · 2015-05-22T19:26:27Z

~~Also because it has both name errors and just does not work as-is (I did this on the train and pushed when I hit the station).~~

tacaswell · 2015-05-22T19:51:03Z

Fixed and force pushed.

tacaswell · 2015-05-22T19:52:05Z

@tritemio Can you see if this fixes your problem? If you need this to work for a class very soon we can work out a monkey-patch recipe.

tritemio · 2015-05-22T20:47:30Z

@tacaswell, thanks for fixing this. I have problems building matplotlib on windows. I tried to build a conda package from this recipe here, replacing the initial part of meta.yml with:

package:
  name: matplotlib
  version: 1.5fix

source:
    git_url: https://github.com/tacaswell/matplotlib
    git_tag: fix_nbagg_close

but it fails with:

C:\Users\laser2002j\AppData\Local\Programs\Common\Microsoft\Visual C++ for Python\9.0\VC\Include\xlocale(342) :
warning C4530: C++ exception handler used, but unwind semantics are not enabled. Specify /EHsc
c:\users\laser2002j\anaconda\conda-bld\work\src\ft2font.h(7) : fatal error C1083: Cannot open include file: 'stdint.h': No such file or directory
error: command 'C:\\Users\\laser2002j\\AppData\\Local\\Programs\\Common\\Microsoft\\Visual C++ for Python\\9.0\\VC\\Bin\\amd64\\cl.exe' failed with exit status 2

Building from sources with python setup.py build fails becaus it cannot find freetype and libpng (that are installed through conda). I guess I will try under linux for now.

tritemio · 2015-05-22T22:29:14Z

@tacaswell, I tested your patch on linux. It solves this specific problem, i.e. that no new figure is shown after closing a figure with the "x" button.

However, I have other issues with the notebook backend on multiple re-executions of the same cell containing a plot. I don't know if these issues are related. If not, I can move these to a new issue:

Procedural API: after the second execution of the same cell no figure is displayed (until the next close() call).
OO API: after each execution, the old fig and ax objects are overwritten, a new figure is created and the RAM is filled with invisible figures. Even plt.close('all') does not free any RAM.

tacaswell · 2015-05-22T23:03:00Z

Those should be new issues. They are related to the issue is the single
ton nature if the display area in the notebook and the way that ipython
captures refs to outputs.

On Fri, May 22, 2015, 18:29 Antonino Ingargiola [email protected]
wrote:

@tacaswell https://github.com/tacaswell, I tested your patch on linux.
It solves this specific problem, i.e. that no new figure is shown after
closing a figure with the "x" button.

However, I have other issues with the notebook backend on multiple
re-executions of the same cell containing a plot. I don't know if these
issues are related. If not, I can move these to a new issue:

Procedural API: after the second execution of the same cell no figure
is displayed (until the next close() call).
2.

OO API: after each execution, the old fig and ax objects are
overwritten, a new figure is created and the RAM is filled with invisible
figures. Even plt.close('all') does not free any RAM.

—
Reply to this email directly or view it on GitHub
#4456 (comment)
.

tacaswell · 2015-07-05T06:47:44Z

This needs to be merged soon.

efiring · 2015-07-05T08:00:02Z

lib/matplotlib/backends/backend_nbagg.py

@@ -287,6 +296,7 @@ def on_message(self, message):
        message = json.loads(message['content']['data'])
        if message['type'] == 'closing':
            self.on_close()
+            self.manager.canvas.close_event()


Just curious: why is this down here instead of inside the on_close() method?

pelson · 2015-07-05T14:27:19Z

I'm not 100% of the behaviour that is being targeted here. Is this fixing a bug, or changing the behaviour? If the latter, we certainly would want a UAT for this.

This needs to be merged soon.

I'm confident we can do this by next weekend - is that too late?

tacaswell · 2015-07-05T15:15:02Z

The problem is that when you hit the red 'x' the figure is closed on the js side, but not on the mpl side so plt.gca() still points to an axes which is not being displayed.

OceanWolf · 2015-07-05T15:40:01Z

It looks fine to me now, MEP27 will fix this anyway, but as the main change occurs only in new_figure_manager_given_figure which MEP27 doesn't even use, and the only other change inCommSocket looks logical, something I will perhaps need for MEP27.

Note I say this having never worked with NbAgg, only WebAgg, but as far as I understand NbAgg just extends WebAgg...

mdboom · 2015-08-06T21:01:23Z

This makes sense on first read and I definitely see the problem it's trying to address. But as the last "nbagg" expert I'll defer to @pelson for the final word on this one.

tacaswell · 2015-08-06T21:13:50Z

Coupled with #4841

tacaswell · 2015-08-16T04:11:01Z

There are two remaining subtle things

make sure figures get closed on kernel restart (also does not happen in 1.4.3)
still get a 'no such comm' error from closing figure programtically from python side

and updating the UAT

jenshnielsen · 2015-08-16T10:46:01Z

Great improvements. Do you want to fix the remaining issues on this branch before merging or do that in a new PR?

I noticed that the connection info in matplotlib.backends.backend_nbagg.connection_info() seems to be wrong in interactive mode enabled with the magic.

I.e. with one figure open and shown it reports the figure a pending show:

print(matplotlib.backends.backend_nbagg.connection_info())
Figure 1 - Figure 1
Figures pending show: 1

As far as I can tell it reports all figures as pending show in interactive mode. This is however not an issue with this branch but already happens on master.

In IPython/Jupyter 4 %matplotlib notebook raises a ShimWarning. We should probably import the comm from ipykernel if IPython 4 is installed.

/usr/local/lib/python2.7/site-packages/IPython/kernel/__init__.py:13: ShimWarning: The `IPython.kernel` package has been deprecated. You should import from ipykernel or jupyter_client instead.
  "You should import from ipykernel or jupyter_client instead.", ShimWarning)

tacaswell · 2015-08-16T11:58:18Z

I think both of those can be done in a different PR, but I want to update the UAT before we merge this.

I don't think that the connection_info is useful in interactive mode. It is looking at _pylab_helpers._activeQue which is a list of open figures which the 'active' one last. When a new figure is set active it is bubbled to the end. It looks like the point of this is so that when the user closes the active figure we can select the most recently active one (should probably implement this as a heap, but). In backend_nbagg.Show in non-interactive mode reaches is and removes the managers from _activeQue (which no other backend does) on show and then uses the length of _activeQue to determine how many figures are pending.

jenshnielsen · 2015-08-16T13:08:51Z

👍

tacaswell · 2015-08-16T14:54:54Z

This seems to have a bad interaction with some styles, but not all, where
it gets stuck in an infinite redraw loop.

On Sun, Aug 16, 2015, 9:08 AM Jens Hedegaard Nielsen <
[email protected]> wrote:

[image: 👍]

—
Reply to this email directly or view it on GitHub
#4456 (comment)
.

tacaswell · 2015-08-17T01:04:34Z

I can't reproduce the issue I was having on this branch, but saw it on a really hacked up branch (basically merged all the outstanding feature PRs) that I have a demo at pygotham off of.

OceanWolf · 2015-08-17T06:10:18Z

lib/matplotlib/backends/backend_nbagg.py

-                                       manager.web_sockets))
-    result.append('Figures pending show: {0}'.format(len(Gcf._activeQue)))
+                                          "Figure {0}".format(manager.num)),
+                                         manager.web_sockets))


I assume you did this as a PEP8 change, but the second line still lacks a space of indent...

OceanWolf · 2015-08-17T06:19:00Z

UAT? What does that mean?

Does your (@tacaswell) comment about _pylab_helpers._activeQue, explain the "careful rebuilding of the activeQue" comment in _pylab_helpers?

jenshnielsen · 2015-08-17T08:00:34Z

User acceptance testing, In the notebook modified in the last commit

jenshnielsen · 2015-08-17T12:08:32Z

Looks like it now needs a rebase otherwise good

tacaswell · 2015-08-17T14:21:26Z

This is a conflict in the UAT, hurray!

Add callback to destroy figure when the user hits the red x in the notebook. This is not the correct solution, but it (might) work for now.

- only actually closes the figure (in the mpl sense) if all of the managers active comms are dead - closes figure when all visible copies are removed from dom - eliminates many of the [IPKernelApp] ERROR | No such comm: XXXX errors closes matplotlib#4841

In interactive mode the number of pending figures will always be equal to the number of open figures. Only include this count in `connection_info` in non-interactive mode.

- update to use python3 - update to use nbformat v4 - add test for close figure on remove from DOM

FIX : first pass at fixing nbagg close issue

jenshnielsen · 2015-08-17T16:45:50Z

I made an issue to track the remaining figure close issues above #4940

tacaswell · 2015-08-17T16:54:14Z

Thanks for taking care of the that issue.

On Mon, Aug 17, 2015, 12:45 PM Jens Hedegaard Nielsen <
[email protected]> wrote:

I made an issue to track the remaining figure close issues above #4940
#4940

—
Reply to this email directly or view it on GitHub
#4456 (comment)
.

tacaswell added the status: needs review label May 22, 2015

tacaswell force-pushed the fix_nbagg_close branch from 78aa7a5 to f08e811 Compare May 22, 2015 19:50

tacaswell added this to the Color overhaul milestone May 22, 2015

tritemio mentioned this pull request May 22, 2015

nbagg backend: multiple execution of same cell with plots #4457

Closed

tacaswell force-pushed the fix_nbagg_close branch from f08e811 to c0cb5b6 Compare July 5, 2015 06:47

tacaswell modified the milestones: next point release, Color overhaul Jul 5, 2015

efiring reviewed Jul 5, 2015
View reviewed changes

tacaswell modified the milestones: next point release, proposed next point release Jul 12, 2015

tacaswell modified the milestones: next point release, proposed next point release Jul 22, 2015

tacaswell changed the title ~~FIX : first pass at fixing nbagg close issue~~ FIX : first pass at fixing nbagg close issue [WIP, don't merge] Aug 15, 2015

tacaswell force-pushed the fix_nbagg_close branch from 7228163 to 7888942 Compare August 16, 2015 04:03

tacaswell changed the title ~~FIX : first pass at fixing nbagg close issue [WIP, don't merge]~~ FIX : first pass at fixing nbagg close issue Aug 16, 2015

tacaswell force-pushed the fix_nbagg_close branch from 7888942 to 2ca678c Compare August 16, 2015 04:07

OceanWolf reviewed Aug 17, 2015
View reviewed changes

jenshnielsen mentioned this pull request Aug 17, 2015

NBAgg: fix Jupyter shim warning #4939

Merged

tacaswell added 5 commits August 17, 2015 10:22

FIX : first pass at fixing nbagg close issue

bd0a3f8

Add callback to destroy figure when the user hits the red x in the notebook. This is not the correct solution, but it (might) work for now.

FIX: move logic where it should be

b16fcf9

WIP: race condition?

6678e02

MNT: suppress pending figure count in interactive

f1c9d78

In interactive mode the number of pending figures will always be equal to the number of open figures. Only include this count in `connection_info` in non-interactive mode.

tacaswell force-pushed the fix_nbagg_close branch from f2e1443 to f1c9d78 Compare August 17, 2015 14:49

TST: update UAT

3aa3909

- update to use python3 - update to use nbformat v4 - add test for close figure on remove from DOM

jenshnielsen mentioned this pull request Aug 17, 2015

NBAgg figure management issues #4940

Closed

3 tasks

jenshnielsen added a commit that referenced this pull request Aug 17, 2015

Merge pull request #4456 from tacaswell/fix_nbagg_close

3520e94

FIX : first pass at fixing nbagg close issue

jenshnielsen merged commit 3520e94 into matplotlib:master Aug 17, 2015

tacaswell removed the status: needs review label Aug 17, 2015

tacaswell deleted the fix_nbagg_close branch October 29, 2015 16:08

Uh oh!

FIX : first pass at fixing nbagg close issue #4456

FIX : first pass at fixing nbagg close issue #4456

Uh oh!

Conversation

tacaswell commented May 22, 2015

Uh oh!

efiring commented May 22, 2015

Uh oh!

tacaswell commented May 22, 2015

Uh oh!

tacaswell commented May 22, 2015

Uh oh!

tacaswell commented May 22, 2015

Uh oh!

tacaswell commented May 22, 2015

Uh oh!

tritemio commented May 22, 2015

Uh oh!

tritemio commented May 22, 2015

Uh oh!

tacaswell commented May 22, 2015

Uh oh!

tacaswell commented Jul 5, 2015

Uh oh!

efiring Jul 5, 2015

Choose a reason for hiding this comment

Uh oh!

tacaswell Jul 5, 2015

Choose a reason for hiding this comment

Uh oh!

pelson commented Jul 5, 2015

Uh oh!

tacaswell commented Jul 5, 2015

Uh oh!

OceanWolf commented Jul 5, 2015

Uh oh!

mdboom commented Aug 6, 2015

Uh oh!

tacaswell commented Aug 6, 2015

Uh oh!

tacaswell commented Aug 16, 2015

Uh oh!

jenshnielsen commented Aug 16, 2015

Uh oh!

tacaswell commented Aug 16, 2015

Uh oh!

jenshnielsen commented Aug 16, 2015

Uh oh!

tacaswell commented Aug 16, 2015

Uh oh!

tacaswell commented Aug 17, 2015

Uh oh!

OceanWolf Aug 17, 2015

Choose a reason for hiding this comment

Uh oh!

OceanWolf commented Aug 17, 2015

Uh oh!

jenshnielsen commented Aug 17, 2015

Uh oh!

jenshnielsen commented Aug 17, 2015

Uh oh!

tacaswell commented Aug 17, 2015

Uh oh!

jenshnielsen commented Aug 17, 2015

Uh oh!

tacaswell commented Aug 17, 2015

Uh oh!

Uh oh!