Acking the receiver (to read the message) of a completed write operation

[LiSu]

It seems like read and write are independent operations in tensorpipe. E.g., if a sender and a receiver are writing into and reading from a specific pipe at the same time, it is possible that the receiver may read nothing if the read operation happens before the write. Is this correct? If so, to guarantee that a read operation can always read something from the pipe, we have to somehow acknowledge the receiver (e.g., using a poller) about the completed write?
@lw

[lw]

I'm not sure I follow. Is there a specific issue you're facing?

[LiSu]

I'm not sure I follow. Is there a specific issue you're facing?

Thanks for the quick reply! The background is that we are trying to build a RPC framework using tensorpipe. The problem we are trying to figure out is: when reading from a pipe, if there is no message in the pipe (e.g., nobody writes to the pipe yet), will the reader be blocked until there is a message in the pipe, or the reader will return immediately and does nothing. Hope this can clarify the question.

[LiSu]

Hi Luca @lw, I think we found the answer for the above question from your comment in issue 405
:

These two sides are used as ringbuffers. When a side wants to send something to the other one, it copies the data from the source user buffer to its outbox, then it does a RDMA copy over InfiniBand to the remote's inbox, and this operation "notifies" the remote (which is continuously polling for events), which then copies it from its inbox to the target buffer (and notifies the sender that the inbox is now empty again and can be overwritten).

A follow-up question is that, besides ibv, tensorpipe also supports uv and shm. In the case of uv and shm, is there a similar polling mechanism like ibv to notify the arrival of data in the remote side?

[lw]

when reading from a pipe, if there is no message in the pipe (e.g., nobody writes to the pipe yet), will the reader be blocked until there is a message in the pipe, or the reader will return immediately and does nothing

The reader will be blocked until something arrives. In practice, the readDescriptor callback will only be invoked once a message arrives.

In the case of uv and shm, is there a similar polling mechanism like ibv to notify the arrival of data in the remote side?

Yes, all transports behave semantically identically, you don't need to worry about the implementation details.