Great, I will try to have a look at this soon.

I think I will not reorganize the channels to not be threads in this PR - it seems a bit out of scope.

Looks like I'm almost there. The only thing remaining is to finish up the autorestart work started in the parent PR.

I am not confident in the in-process kernel - the test coverage is minimal, and there are zero use cases within IPython. I really don't like that we have something this complicated that is totally unused and almost totally untested.

I think this is ready for review

I would like to help here because I think this could simplify things a lot for us. How can I do it?

Mainly this just needs review, I think it's complete now.​

I meant help with testing :) Is there something expected to work differently with the new API? Some corner case that needs a little bit of extra help to test? If not, then just ignore me :)

Gotcha, thanks.

Most things work the same, with these changes:

1. if you are starting a kernel, create a KernelManager
2. if you want to use the kernel, create a KernelClient (this means that to connect to an already-running Kernel, you do not create a KernelManager)
3. shell_channel.methods are all available as KernelClient.methods (this is just a proxy for convenience, the old way works as well)
4. kernels can autorestart, rather than relying on user-input after heartbeat failures. To enable this behavior, create a KernelManager(...autorestart=True). autorestart is not the default behavior.

Testing restarts in qtconsole (no notebook):

I ran the qtconsole and tried killing the kernels using:

import sys
from ctypes import CDLL
# This will crash a Linux or Mac system; equivalent calls can be made on Windows
dll = 'dylib' if sys.platform == 'darwin' else 'so.6'
libc = CDLL("libc.%s" % dll) 
libc.time(-1)  # BOOM!!

I did this repeatedly and in different tabs. I observed the following tracebacks:

[IPythonQtConsoleApp] kernel restarted
Traceback (most recent call last):
  File "/Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/frontend/qt/base_frontend_mixin.py", line 138, in _dispatch
    handler(msg)
  File "/Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/frontend/qt/console/ipython_widget.py", line 179, in _handle_execute_reply
    super(IPythonWidget, self)._handle_execute_reply(msg)
  File "/Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/frontend/qt/console/frontend_widget.py", line 423, in _handle_execute_reply
    self._request_info['execute'].pop(msg_id)
KeyError: u'387e4e3e-4386-4f06-94b9-6de3dd911f7d'

Traceback (most recent call last):
  File "/Users/bgranger/Documents/Computing/IPython/code/ipython/IPython/frontend/qt/console/mainwindow.py", line 241, in close_tab
    kernel_manager.shutdown_kernel()
NameError: global name 'kernel_manager' is not defined

I am trying to find reproducible steps to help debug this.

I am having trouble finding a good way to reproduce these errors, but the second one is a clear bug in the code. The kernel_manager variable is simply not defined in that scope of mainwindow.py.

Great work, merging!