How zero copy handles variable length data under same topic

[xixioba]

Is there an example where this can be achieved :)

[JLBuenoLopez]

@xixioba,

Zero copy has some constraints one of them being that the type must be bounded (so no variable length is currently allowed). You can find more information in the corresponding section of Fast DDS documentation

[xixioba]

Are there some plans to implement this feature?

[JLBuenoLopez]

It is not currently in the Fast DDS roadmap. You might contact our support team if you are interested in a feature acceleration commercial estimation.

[ZhenshengLee]

@xixioba Hi

this ros2 design article of zero-copy talked about loaned_api with non-POD messages, which might help you.

[xixioba]

Ok, I understand that protobuf serialization and deserialization take a considerable amount of time, maybe I will try capnproto, in fact I completed the test comparison through fastdds, capnp is indeed much faster, but unfortunately, I have to put the data in idl fixed size structure to enable zero copy, it can only be used for testing

[ZhenshengLee]

Technically, there is a design.

• For inter-machine-comm, use protobuf, for example gRPC in application layer.
• (In the edge between inter-machine and intra-machine)Create a bridge between protobuf and rosidl, for example https://github.com/ipa-rar/gRPC-ros-bridge , note that the bridge will add many times of copy, so the bridge must not be in the Critical Chain of the latency propagation chain.
• For intra-machine-comm, use fixed-size-idl with zero-copy to ensure the latency meeting performance requirement

[xixioba]

yes，I understand that this is the currently achievable solution
Also, I looked at zenoh implementation and it seems that it can meet some of my needs zero copy and even communicate with existing DDS, I also see that you are involved, what do you say about zenoh's latency and load

[ZhenshengLee]

I didn't look it indepth, but I know there is a project from AutoCore(based on Nanjing, China) that would be good materials which is https://github.com/autocore-ai/zenoh_flow_autoware.

BTW, would you mind contacting me via wechat: zhensheng_li so we can discuss it more with Chinese community?

[xixioba]

ok, looks cool

[ZhenshengLee]

Also, the mechanism of zero copy of eCAL in https://continental.github.io/ecal/advanced/layers/shm.html#zero-copy-mode-optional does not require the bounded-size data types. Which may help.

[xixioba]

Thanks, I also tested ecal, unfortunately, it's not fully compatible with macos, even the pub/sub example doesn't work, ref https://github.com/continental/ecal/issues/697#issuecomment-1170668403, I'll try it again

[ZhenshengLee]

this issue eclipse-iceoryx/iceoryx#911 describe an idea of dynamic size type support in iceoryx, which may help, but this has not been implemented yet.

[xixioba]

Thanks, I'm interested in eclipse-iceoryx, but when I test cyclonedds with iceoryx (on macos), it sometimes crashes, since iox-roudi needs to be run separately and it's difficult for me to manage, I went over the related issue eclipse-iceoryx/iceoryx#970, maybe it works Feel free to use iceoryx, if you have experience I would really appreciate it