IPoIB: Fix TX queue lockup with mixed UD/CM traffic

Eli Cohen · Roland Dreier · commit f0dc117abdfa · 2010-03-11T13:37:11.000-08:00
The IPoIB UD QP reports send completions to priv-&gt;send_cq, which is
usually left unarmed; it only gets armed when the number of
outstanding send requests reaches the size of the TX queue. This
arming is done only in the send path for the UD QP.  However, when
sending CM packets, the net queue may be stopped for the same reasons
but no measures are taken to recover the UD path from a lockup.

Consider this scenario: a host sends high rate of both CM and UD
packets, with a TX queue length of N.  If at some time the number of
outstanding UD packets is more than N/2 and the overall outstanding
packets is N-1, and CM sends a packet (making the number of
outstanding sends equal N), the TX queue will be stopped.  When all
the CM packets complete, the number of outstanding packets will still
be higher than N/2 so the TX queue will not be restarted.

Fix this by calling ib_req_notify_cq() when the queue is stopped in
the CM path.

Signed-off-by: Eli Cohen &lt;eli@mellanox.co.il&gt;
Signed-off-by: Roland Dreier &lt;rolandd@cisco.com&gt;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -752,6 +752,8 @@ void ipoib_cm_send(struct net_device *dev, struct sk_buff *skb, struct ipoib_cm_
 		if (++priv->tx_outstanding == ipoib_sendq_size) {
 			ipoib_dbg(priv, "TX ring 0x%x full, stopping kernel net queue\n",
 				  tx->qp->qp_num);
+			if (ib_req_notify_cq(priv->send_cq, IB_CQ_NEXT_COMP))
+				ipoib_warn(priv, "request notify on send CQ failed\n");
 			netif_stop_queue(dev);
 		}
 	}

Original file line number	Diff line number	Diff line change
`@@ -752,6 +752,8 @@ void ipoib_cm_send(struct net_device dev, struct sk_buff skb, struct ipoib_cm_`
`752`	`752`	`if (++priv->tx_outstanding == ipoib_sendq_size) {`
`753`	`753`	`ipoib_dbg(priv, "TX ring 0x%x full, stopping kernel net queue\n",`
`754`	`754`	`tx->qp->qp_num);`
	`755`	`+ if (ib_req_notify_cq(priv->send_cq, IB_CQ_NEXT_COMP))`
	`756`	`+ ipoib_warn(priv, "request notify on send CQ failed\n");`
`755`	`757`	`netif_stop_queue(dev);`
`756`	`758`	`}`
`757`	`759`	`}`