IPEP 21: widget messages #4195

minrk · 2013-09-11T22:41:11Z

IPEP 21 has full description of the proposal.

This is just the basic communication tube, through which widgets might communicate.

It's pretty basic, but it seems to work reasonably well.

Some example use cases for the Comms implemented here: http://nbviewer.ipython.org/6547109

This also refactors callbacks in a clearer way, allowing callback cleanup since widget messages dramatically exacerbate #2524.

closes #2524.

stevengj · 2013-09-12T01:37:47Z

cc: @dcjones, who has been implementing JS-based plotting widgets in IJulia.

Would be nice in the PEP to refer to the "kernel" rather than "Python" side, and to separate the things that any kernel must do from the specific implementation details in the IPython kernel.

minrk · 2013-09-12T05:01:11Z

Would be nice in the PEP to refer to the "kernel" rather than "Python" side

I tried to do this, I will read through again and see where I slipped up. There are actually three pieces here:

message spec (Kernel and Frontend)
Python API implementing the spec
Javascript API implementing the spec

minrk · 2013-09-12T05:07:34Z

I read through the IPEP, and Python is never mentioned in the spec - I only use Kernel and Frontend there. When I switch gears to talk about the specific APIs of the Python and Javascript implementations, I discuss the languages in which those implementations exist. I have clarified the transition, though.

ellisonbg · 2013-09-12T21:01:57Z

Great work, I love the symmetry and simplicity!

Widget Creation:

Should clarify how the widget id is selected - kernel picks.
Do the two sides share the widget id?
Clarify what the frontend should do when it receives the widget_create message
What happens with the frontend gets a widget_create message but doesn't know about that widget type? Seems like the frontend should send a widget_destroy message.
Do we allow the frontend to send the widget_create message?

Widget object:

I think we should try the simple design - just have data and not the full message. The data part is general enough to really cover everything. The only case where I could imagine we would want to include the metadata is if we started to define the details of the data JSON object to include things like our display messages.

ellisonbg · 2013-09-12T21:05:06Z

I think the biggest question I have at this point is how widgets relate to our standard display messages. What happens if a widget needs to update the state of a matplotlib plot? Seems like there needs to be some coordination there. Do widgets handle the display messages? Do why manage their own OutputArea?

Another question is if we are going to have any notion of widget nesting. If there is a plot that updates with the values of 4 sliders and 2 checkboxes, is that 1 widget or 6?

ellisonbg · 2013-09-12T21:09:48Z

IPython/html/static/notebook/js/widget.js

+        var msg_types = ['widget_create', 'widget_destroy', 'widget_update'];
+        for (var i = 0; i < msg_types.length; i++) {
+            var msg_type = msg_types[i];
+            kernel.register_iopub_handler(msg_type, $.proxy(this[msg_type], this));


Do we make sure that we unregister these handlers when the widget is destroyed?

These are only registered once - there is only one WidgetManager.

ellisonbg · 2013-09-12T21:14:00Z

In the IPEP, we should emphasize that the widget_create and widget_destroy messages mean that the receiver should do those actions. Part of what is a bit confusing with a fully symmetric protocol is preventing infinite loops of messages. To prevent this, we should be very clear that the handle methods should never send the same message type it is handling.

ellisonbg · 2013-09-12T21:14:29Z

Could you point to your examples?

ellisonbg · 2013-09-12T21:17:12Z

@ivanov and @jdfreder I think the best way for us to test this widget messages is to use it to write our higher level JS/Python APIs. You are the two leads on that (Paul on the Python side, Jon on the Javascript side). Can you both start to think about that stuff. I am more than will to have design discussion with both of you. I want us to start this stuff as soon as possible so we have time to iterate and bake the design before 2.0.

jdfreder · 2013-09-12T21:19:14Z

@ellisonbg sounds like a good idea. When are you going to be back in town?

jasongrout · 2013-09-12T21:26:28Z

In the PEP, you ask: "Question: the handle_foo messages get the data dict, not the full message. This means they don't have access to metadata, etc. Should the handlers get the full message instead?"

I think yes. It would be useful, for example, to access the metadata. It might also be useful to see the timestamp when a message was generated, for example. And at least passing in the metadata would be more consistent with other handlers, right?

ellisonbg · 2013-09-12T21:37:11Z

@jdfreder I will be back Sat night, back in the office on Monday. I will be in touch.

ellisonbg · 2013-09-12T21:37:58Z

@jasongrout makes a good point. I could imagine that for some widgets, the timestamps are important - they could be used to throttle the update rates.

ellisonbg · 2013-09-12T21:39:37Z

I should mention - I play some last week with some slider widgets and quickly found that it was easy to flood the kernel with calls. I am wondering if we want to add code on the JS side that can automatically throttle the rate at which the update messages are send. I think this is going to be a really common problem that most highly interactive widgets have.

jasongrout · 2013-09-12T21:42:47Z

Another question to consider: How do you lay out multiple widgets? Suppose I want two sliders next to each other, above an interactive plot, with two buttons off to the side? I can construct the necessary html, for example. How do I specify where to put the widgets in the html? This is related to @ellisonbg's comment about nested widgets.

For Sage interacts, we have 3 layout systems, two of which are currently active. One is a list of lists (i.e., rows of columns). Another that might be more applicable here is to just spit out the html, then give the "locations", which are jquery selectors for the places to put the widgets.

minrk · 2013-09-12T21:52:30Z

An update from the dev meeting and today's discussion:

These are not widgets, they are the basic communication object used to implement widgets. As a result, they should not be called widgets, but something else. I think I'm going with Comm for now. When you create a Widget, it will use a Comm, but it will not be a Comm.

Widget Creation:

Should clarify how the widget id is selected - kernel picks.

The creator picks the ID, can be done from either side.

Do the two sides share the widget id?

Yes

Clarify what the frontend should do when it receives the widget_create message
What happens with the frontend gets a widget_create message but doesn't know about that widget type?
Seems like the frontend should send a widget_destroy message.

Right now, it just logs an error. widget_destroy isis probably the right thing to do.

Do we allow the frontend to send the widget_create message?

Yes, the spec is fully symmetrical. I initially didn't allow this, because of a note from the dev meeting,
but it is actually more complicated to add the asymmetry.

Widget object:

I think yes. It would be useful, for example, to access the metadata.

Okay, I will pass along the whole message - it's not appreciably more complicated.

In the IPEP, we should emphasize that the widget_create and widget_destroy messages mean that the receiver should do those actions.

Will do.

Part of what is a bit confusing with a fully symmetric protocol is preventing infinite loops of messages. To prevent this, we should be very clear that the handle methods should never send the same message type it is handling.

I don't think so. Other than creation / destruction, there is only one message type, so if message handlers should ever send a message back at all, this restriction cannot be enforced. It makes perfect sense for an update from one side to result in a different update coming back (e.g. changing a value causing a replot). It is up to the implementer to ensure they don't get stuck.

minrk · 2013-09-12T21:53:38Z

@jasongrout - this is not GUI widgets, only the communication part. Layout, etc. is entirely independent of this.

minrk · 2013-09-12T21:54:25Z

I play some last week with some slider widgets and quickly found that it was easy to flood the kernel with calls. I am wondering if we want to add code on the JS side that can automatically throttle the rate at which the update messages are send. I think this is going to be a really common problem that most highly interactive widgets have.

Yes, I ran into this myself, and quickly started throttling requests. I'm not sure how to do it in a generic way that we can reasonably enforce on all widget authors, though.

ellisonbg · 2013-09-12T21:55:03Z

@minrk this brings up a good point about naming. The current naming
implies that there will be a 1-to-1 mapping between Widget objects and UI
controls, but that is definitely not the case. A single widget object
could manage 100 UI controls. Will have to think more about the naming of
things.

On Thu, Sep 12, 2013 at 2:53 PM, Min RK [email protected] wrote:

@jasongrout https://github.com/jasongrout - this is not GUI widgets,
only the communication part. Layout, etc. is entirely independent of this.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4195#issuecomment-24359014
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
[email protected] and [email protected]

jasongrout · 2013-09-12T21:57:44Z

@minrk: it makes sense that layout issues are above the communication layer. But the creation handler will have to address this somehow, so that's another vote for the whole message (including maybe layout metadata) being passed into the create handler.

minrk · 2013-09-12T21:58:29Z

I think the biggest question I have at this point is how widgets relate to our standard display messages. What happens if a widget needs to update the state of a matplotlib plot? Seems like there needs to be some coordination there. Do widgets handle the display messages? Do why manage their own OutputArea?

That seems like it would be up to the Widget implementation - some might hook up events on an existing OutputArea, some might manipulate their own div elsewhere (sidebar namespace browser). It doesn't seem like a question that can have a generic answer.

Another question is if we are going to have any notion of widget nesting. If there is a plot that updates with the values of 4 sliders and 2 checkboxes, is that 1 widget or 6?

I don't know. I also am not sure this has any bearing on the code here - this is just the very basic communication code for allowing some object in the Kernel to talk to some object in the Frontend and vice versa.

jasongrout · 2013-09-12T22:03:28Z

@minrk: if you're changing the names to reflect a message channel, maybe instead of update it should be send.

minrk · 2013-09-12T22:06:22Z

@jasongrout reading my mind - I have switched create/update/destroy to more comm-appropriate open/send/close.

jasongrout · 2013-09-12T22:13:57Z

I like the simplicity of the open/send/close/handlers idea. Basically you are just creating a private message channel on top of the existing messaging framework. I can see how just passing the data attribute makes sense---the IPython message wrapping the data attribute is nothing but routing information.

minrk · 2013-09-12T22:29:45Z

I can see how just passing the data attribute makes sense---the IPython message wrapping the data attribute is nothing but routing information.

That's true - and if the entity constructing the object is interesting in timings, it can always add them to the data messages. I guess I will live it using just data for now, but we can think a bit more carefully later on about handing off the whole message. It is not at all a difficult change to make.

jasongrout · 2013-09-12T22:44:37Z

So how do you handle a channel open message from a frontend to a kernel that has multiple frontends? Do you broadcast out an open message to all other frontends (like you would echo input)? What about channel close messages? Are send messages from the kernel sent to all frontends?

minrk · 2013-09-12T22:49:08Z

So how do you handle a channel open message from a frontend to a kernel that has multiple frontends? Do you broadcast out an open message to all other frontends (like you would echo input)? What about channel close messages? Are send messages from the kernel sent to all frontends?

This is a good point, where there is an asymmetry - one kernel, possibly multiple frontends. It seems to me that it makes sense in general for a Comm on the Kernel side to re-broadcast both open and destroy messages that it has received. Then the only handling needed on the Frontend-side is to ignore open and close messages for Comms it has already opened or closed. I don't think that same generalization can be made for the 'normal' messages in between open and close, and that decision should be left to widget implementers.

Carreau · 2013-09-13T08:02:50Z

IPython/html/static/notebook/js/comm.js

+ * @submodule comm
+ */
+
+var IPython = (function (IPython) {


Would you like to start using require ?

I would like to use require but that should be a separate PR.

ellisonbg · 2013-09-13T16:58:18Z

A few more thoughts:

I thought about the name more and I am liking the name Channel instead of Comm. I like the open, close, send names though.
With our renames, these classes really become focused on communications. However, the handle_send method will presumably have all of the view logic for our "widgets" (here I am using that name to include the frontend UI side of things as well. IOW, I think it is important to cleanly separate the model from the view, but with the current design, the handle_send method would have the view logic.
I think it is important to have an object that is bound to a particular cell and has a handle on that cells DOM elements for view manipulation. Is that object the Comm/Channel object? Is that a new View object? Are Comm/Channel objects bound to cells?

for reusing default execute callbacks

Javascript-side version for creating and connecting Comms in one call Without a `get_ipython()`-like global handle, Comm constructor can't do the same magic as the IPython one.

minrk · 2013-10-23T21:39:28Z

I think this is ready to go, if anyone wants to have a last look.

jasongrout · 2013-10-24T15:27:52Z

+1 to merging.

IPEP 21: widget messages

jasongrout · 2013-10-24T19:55:32Z

Woohoo! Great work!

ellisonbg · 2013-10-24T19:59:41Z

Awesome!

On Thu, Oct 24, 2013 at 12:55 PM, Jason Grout [email protected]:

Woohoo! Great work!

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/4195#issuecomment-27024534
.

Brian E. Granger
Cal Poly State University, San Luis Obispo
[email protected] and [email protected]

SylvainCorlay · 2013-10-24T20:35:25Z

Great. Thank you.

fperez · 2013-10-24T21:09:19Z

Fantastic job, guys. Very, very happy about this.

Carreau · 2013-10-26T11:19:08Z

@minrk in this, kernel.js you changed object_info_request to object_info. it broke tooltip, and is an API change (imho) what should we do about ?

(I should have seen it, sorry)

comm merging renamed object_info_request to object_info

minrk · 2013-10-26T17:44:08Z

Must have been mixed up as part of the rebase, I know I tested tooltips. It was wrong to use _request in one method name, where we leave it off of the others (execute, complete).

IPEP 21: widget messages

comm merging renamed object_info_request to object_info

ellisonbg reviewed Sep 12, 2013
View reviewed changes

Carreau reviewed Sep 13, 2013
View reviewed changes

minrk added 14 commits October 22, 2013 19:44

move comm.js to kernel service

f52bb4d

attach comm_manager to kernel

2227b13

make parent_header available from the Shell object

5bd8c19

set parent header properly in comm messages

99da4b6

allow metadata in Kernel-side Comm messages.

360345a

add CodeCell.get_callbacks

a9d099c

for reusing default execute callbacks

add unregister_target to CommManagers

a9a6bd0

log exceptions in Comm handlers

9ce5d2b

add target_name as first arg to Comm constructor

6521d1b

Add CommManager.new_comm

2afefa0

Javascript-side version for creating and connecting Comms in one call Without a `get_ipython()`-like global handle, Comm constructor can't do the same magic as the IPython one.

catch errors in comm callbacks

c4185b7

allow callbacks on status messages

1da827d

set parent of status messages for comm_msgs

261ea42

don't expose comm_id arg via new_comm

01609ca

minrk added a commit that referenced this pull request Oct 24, 2013

Merge pull request #4195 from minrk/widget-msg

de09adf

IPEP 21: widget messages

minrk merged commit de09adf into ipython:master Oct 24, 2013

Carreau added a commit to Carreau/ipython that referenced this pull request Oct 26, 2013

fix and add shim for change introduce by ipython#4195

bda4afa

comm merging renamed object_info_request to object_info

minrk deleted the widget-msg branch October 26, 2013 17:44

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

Merge pull request ipython#4195 from minrk/widget-msg

4dc6472

IPEP 21: widget messages

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014

fix and add shim for change introduce by ipython#4195

76adc47

comm merging renamed object_info_request to object_info

brandoncurtis pushed a commit to brandoncurtis/sagecell that referenced this pull request Jan 30, 2015

Update to work with IPython comm infrastructure (ipython/ipython#4195)

52ed508

Uh oh!

IPEP 21: widget messages #4195

IPEP 21: widget messages #4195

Uh oh!

Conversation

minrk commented Sep 11, 2013

Uh oh!

stevengj commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg Sep 12, 2013

Choose a reason for hiding this comment

Uh oh!

minrk Sep 12, 2013

Choose a reason for hiding this comment

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

jdfreder commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

ellisonbg commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

jasongrout commented Sep 12, 2013

Uh oh!

minrk commented Sep 12, 2013

Uh oh!

Carreau Sep 13, 2013

Choose a reason for hiding this comment

Uh oh!

ellisonbg Sep 13, 2013

Choose a reason for hiding this comment

Uh oh!

ellisonbg commented Sep 13, 2013

Uh oh!

minrk commented Oct 23, 2013

Uh oh!

jasongrout commented Oct 24, 2013

Uh oh!

jasongrout commented Oct 24, 2013

Uh oh!

ellisonbg commented Oct 24, 2013