Fix a really stupid and old bug.

bombela · bombela · commit 5c66bf083acb · 2014-09-12T23:50:39.000-07:00
- server side, you accept a new channel
 - you stack an heartbeatonchannel on top
 - you slow everything enough so the client already sent some heartbeat
   msgs before you could read the first msg (cpu intensive task,
   time.sleep...)
 - you send back the response
 - you close the heartbeatonchannel
 - this kills the coroutine handling the heartbeat
 - this action eventually switch to the hearbeat coroutine to raise
   GreenletExit in it.
 - which after might ends up to the recver coroutine (not yet killed)
 - the recever coroutine gets the heartbeat msg from the client
 - and stupidly process to spawn a new heartbeat coroutine!
 - this eventually switch back to the close()
 - which set the self._heartbeat to None (so potentially another
   heartbeat msg could spawn another heartbeat coroutine!)
 - after enough back and forth, self._channel is set to None
 - one or more of the heartbeat coroutines is trying to access
   self._channel.emit... and BOOM, you get (an harmless) stacktrace in
   the logs.

The solution is to set a flag when close() is called: self._close =
True.

_start_hearbeat() called from the recver coroutine can then check this
flag.

Last but not least, the heartbeat coroutine checks this flag before
killing the parent coroutine. This is not strictly necessary, but if in
the future the code in close() eventually yields back to gevent (for
example, by trying to kill another coroutine before the heartbeat's
one), this will prevent a LostRemote to be raised on close().

And finally, a forgotten variable was removed (in channel.py).
diff --git a/zerorpc/channel.py b/zerorpc/channel.py
@@ -171,7 +171,6 @@ def __init__(self, channel, inqueue_size=100):
         self._input_queue_reserved = 1
         self._remote_can_recv = gevent.event.Event()
         self._input_queue = gevent.queue.Queue()
-        self._lost_remote = False
         self._verbose = False
         self._on_close_if = None
         self._recv_task = gevent.spawn(self._recver)
diff --git a/zerorpc/heartbeat.py b/zerorpc/heartbeat.py
@@ -37,6 +37,7 @@
 class HeartBeatOnChannel(ChannelBase):
 
     def __init__(self, channel, freq=5, passive=False):
+        self._closed = False
         self._channel = channel
         self._heartbeat_freq = freq
         self._input_queue = gevent.queue.Channel()
@@ -58,6 +59,7 @@ def emit_is_supported(self):
         return self._channel.emit_is_supported
 
     def close(self):
+        self._closed = True
         if self._heartbeat_task is not None:
             self._heartbeat_task.kill()
             self._heartbeat_task = None
@@ -75,13 +77,14 @@ def _heartbeat(self):
                 self._remote_last_hb = time.time()
             if time.time() > self._remote_last_hb + self._heartbeat_freq * 2:
                 self._lost_remote = True
-                gevent.kill(self._parent_coroutine,
-                        self._lost_remote_exception())
+                if not self._closed:
+                    gevent.kill(self._parent_coroutine,
+                            self._lost_remote_exception())
                 break
             self._channel.emit('_zpc_hb', (0,))  # 0 -> compat with protocol v2
 
     def _start_heartbeat(self):
-        if self._heartbeat_task is None and self._heartbeat_freq is not None:
+        if self._heartbeat_task is None and self._heartbeat_freq is not None and not self._closed:
             self._heartbeat_task = gevent.spawn(self._heartbeat)
 
     def _recver(self):
