Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[qp_impl.hpp:131] poll till completion error: 12 transport retry counter exceeded #3

@minghust

Description

@minghust

您好,感谢开源rlib!我在使用过程中,遇到了一个问题:

背景:client端(在机器1)使用线程t1和server端(在机器2)建立2个RCQP连接(QP1,QP2)后,client端的t1线程内创建一个新线程t2。接下来,t1使用QP1对server进行one-sided RDMA READ,t2使用QP2对server进行one-sided RDMA WRITE。t1和t2的RDMA READ/WRITE是并行的(读写没有任何冲突)。

到这里本应该没有问题,但是t2的RDMA WRITE却无法写成功(通过查看server端mem region未被修改而得知),导致t2在poll cq时出现 “transport retry counter exceeded” 报错。

查阅RDMA Aware Networks Programming User Manual (Rev 1.7),该错误的解释是:

6.2.13 IBV_WC_RETRY_EXC_ERR
This event is generated when a sender is unable to receive feedback from the receiver. This
means that either the receiver just never ACKs sender messages in a specified time period, or it
has been disconnected or it is in a bad state which prevents it from responding.

奇怪的是,如果t2使用QP1进行RDMA WRITE,则可以写成功,poll也没问题(注意到QP1和QP2都是使用class RRCQP中的connect函数分2次成功连接的)。

但我并不希望t1和t2共用一个RCQP,因为t1和t2会争抢completion queue,比如t1 poll到了t2的ack,导致t1认为自己的RDMA READ成功了,但实际上可能还没读到remote data。

希望您可以解答,谢谢!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions