DDS discovery from windows linux containers to executable on windows host

[j-p-e]

Is there an already existing issue for this?

not quite - #1698 is relevant but I have more than one docker container so can't map them all to port 7414/7415 on the host

Expected behavior

More of a question than a bug though I guess it would have been nice if it 'just worked' - we are running a simulation system where we want to use a DDS-capable windows executable for graphics acceleration, running against other docker linux containers (Docker Desktop with WSL2 in linux mode on Win10) that also use DDS, with arbitrary comms between them all, but all on the same machine. eProsima has so far been working great when they're all containers, but the new bit is we need the exe separated out from the rest. One caveat is we are tied to v1.9.3 for the foreseeable (might make a jump to latest later on)

When we have them all as linux containers in the same docker network (hosted direct on linux or in docker-ce on WSL2 on windows, or in docker desktop on windows) everything works fine, we don't need to do any special subscriber/publisher network setup. But now the exe will be on host IP, and because they are linux containers they are not allowed to use network:host mode on docker (and I think from reading another issue this might confuse DDS discovery anyway without the newest shared memory stuff) - so when we run with the windows executable+linux containers setup the docker containers don't DDS-discover the windows executable or vice versa, they all just wait for that missing connection. I'm guessing we just need to change our XML and/or do some basic port forwarding or adjustments to docker network setup.

All suggestions welcome

Current behavior

a process on windows host can't dds discover a docker network group of containers

Steps to reproduce

run a dds enabled process on a windows host, docker desktop, and a dds enabled container

Fast DDS version/commit

v1.9.3

Platform/Architecture

Windows 10 Visual Studio 2019

Transport layer

Default configuration, UDPv4 & SHM

Additional context

No response

XML configuration file

No response

Relevant log output

No response

Network traffic capture

No response

[j-p-e]

am potentially open to running docker-ce in WSL instead - this allows me to to set network=host on the containers - at which point the first container sees the exe, but none of the others see either of the first two or each other. I can see on wireshark that the first container and the exe are sending lots of UDP, but don't see anything from the others. I have tried forcing UDP via the XML, and mounting shared memory into the containers

[JLBuenoLopez]

Hi @j-p-e,

I have moved your issue to the corresponding forum. As you state, it is not a proper library bug, besides Fast DDS (previously FastRTPS) 1.9.3 being end of life since last month (see our release support). If you need official support for this version you will have to speak with our support team.

Nevertheless, I am missing some information in your issue description. You are saying that XML files are used to configure your Fast DDS application but no file has been attached. It would also be helpful to have some more detailed steps to reproduce your use case, for instance the way you launch the docker containers. Finally, Shared Memory Transport was included in Fast DDS release 1.10.0 so you should not be concerned with this transport (and consequently it is not possible that you are using this transport as the issue description says).

[j-p-e]

thanks - yes agree there is no shared memory. My starting point for XML (working fine when everything is a container) was to use this for all participants i.e. everything on 7412 and 7413 in their own IP address, however this doesn't work now that I need one node on the host

    <participant profile_name="participant_profile">
        <rtps>
            <builtin>
                <discovery_config>
                    <leaseDuration>
                        <sec>DURATION_INFINITY</sec>
                    </leaseDuration>
                </discovery_config>
            </builtin>
        </rtps>
    </participant> 

The core issue is that the linux containers all discover each other, but the node on the host opens dds, starts listening, and isn't discovered by any of the containers

I have since tried :
port mapping arbitrary host ports to all 7412,7413 ports when running containers (no change)
adding windows host WSL IP, host.docker.internal IP (which are different with a user network), 127.0.0.1, and 0.0.0.0 to defaultUnicastLocatorList in the exe that will run on the host (no change)
adding host.docker.internal IPs with 7412 to metatrafficUnicastLocatorList in the host (no change)
adding initial peers on both sides with every IP I can think of (no change)
telling nodes to use custom UDP transport instead of built in (no change)
moving each linux container to use it's own participant so that host can use 7412,7413 and linux container metatrafficunicast can be at 7414, 7416, 7418, and unicastlocators at 7415, 7317, 7319, and then port mapping when running containers 7414:7414/udp, 7415:7515/udp etc (result : all containers lose connectivity to each other as well as the host)
running with the linux containers on WSL host network instead of user network (no change)
running various portproxy things on host (not useful, I think that only does TCP)
running using docker-ce in WSL rather than docker desktop on host (calling into host for the exe) - no change
when using WSL, adding sudo iptables -A INPUT -i docker0 -j ACCEPT (doesn't seem to do much)

From a linux container, I can nmap one of the PC IPs and see 7412 and 7413 open from the host executable I'm trying to get to, so it is vaguely there. When running with the separated out container port numbers I can also nmap from the host and see all of the containers sitting there nicely on their own ports (however those can't talk to each other at all). So I think I just need discovery. I have not paid particular attention to 7400, or turning multicast off, not sure if I need to do anything with those

[MiguelCompany]

I think initial peers could be useful in this case. Having the participants in the containers forcefully discover the one in the host may be something you could try.

[j-p-e]

with what IP though ? this is becoming more of a docker question but :

from windows PC with host exe running>nmap -sU -p7400-7500 172.18.51.66
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 12:58 GMT Summer Time
Nmap scan report for host.docker.internal (172.18.51.66)
Host is up (0.00036s latency).
Not shown: 98 closed udp ports (port-unreach)
PORT STATE SERVICE
7400/udp open|filtered rtps-discovery
7412/udp open|filtered unknown
7413/udp open|filtered unknown

host can see host DDS, all good so far

docker run --rm -it alpine sh # or docker run --rm --network "bridge" -it alpine sh which is closer to how I'll run it
/ # apk add net-tools nmap
/ # nmap -sU -p7400-7500 host.docker.internal
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 11:56 UTC
Nmap scan report for host.docker.internal (192.168.65.2)
Host is up (0.0016s latency).
All 101 scanned ports on host.docker.internal (192.168.65.2) are in ignored states.
Not shown: 101 open|filtered udp ports (no-response)

/ # nmap -sU -p7400-7500 172.18.51.66
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 11:57 UTC
Nmap scan report for 172.18.51.66
Host is up (0.0019s latency).
All 101 scanned ports on 172.18.51.66 are in ignored states.
Not shown: 101 open|filtered udp ports (no-response)

same result with
docker run --rm -it -p 7412:7412/udp -p 7413:7413/udp alpine sh
or
docker run --rm -it -p 172.18.51.66:7412:7412/udp -p 172.18.51.66:7413:7413/udp alpine sh

letting me do the latter at all is suspicious, since the host side ip:port should conflict with the dds exe

[j-p-e]

one more for good luck
docker run --rm -it --net=host alpine sh # ifconfig gives bridge 172.19.0.1, docker0 172.17.0.1, eth0 192.168.65.3
/ # apk add net-tools nmap
/ # nmap -sU -p7400-7500 172.17.0.1
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 12:53 UTC
Nmap scan report for 172.17.0.1
Host is up (0.0000060s latency).
All 101 scanned ports on 172.17.0.1 are in ignored states.
Not shown: 101 closed udp ports (port-unreach)
Nmap done: 1 IP address (1 host up) scanned in 1.20 seconds

/ # nmap -sU -p7400-7500 host.docker.internal
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 12:54 UTC
Nmap scan report for host.docker.internal (192.168.65.2)
Host is up (0.0014s latency).
All 101 scanned ports on host.docker.internal (192.168.65.2) are in ignored states.
Not shown: 101 open|filtered udp ports (no-response)
MAC Address: F6:16:36:BC:F9:C6 (Unknown)
Nmap done: 1 IP address (1 host up) scanned in 4.56 seconds

/ # nmap -sU -p7400-7500 172.19.0.1
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 12:54 UTC
Nmap scan report for 172.19.0.1
Host is up (0.0000060s latency).
All 101 scanned ports on 172.19.0.1 are in ignored states.
Not shown: 101 closed udp ports (port-unreach)
Nmap done: 1 IP address (1 host up) scanned in 1.21 seconds

/ # nmap -sU -p7400-7500 172.18.51.66
Starting Nmap 7.92 ( https://nmap.org ) at 2022-03-28 12:55 UTC
Nmap scan report for 172.18.51.66
Host is up (0.0017s latency).
All 101 scanned ports on 172.18.51.66 are in ignored states.
Not shown: 101 open|filtered udp ports (no-response)

Nmap done: 1 IP address (1 host up) scanned in 32.35 seconds

[j-p-e]

the docker desktop example for linux containers to host (python http server on host at 8000, no port mappings, curl http://host.docker.internal:8000 from linux container) works fine for all three network modes

[j-p-e]

I've created the problem without DDS so this is a docker question. Running a simple python UDP server on the host and a UDP client on the container works fine in all three network modes (host, default, bridge) with no port mapping, but none of them work the other way around even with the port mapped. So the host has no UDP route to the linux container

[j-p-e]

have found a UDP route from windows host to linux container (using port 7000 as example) by running the container as
docker run -v %CD%:/here --rm -it --publish mode=host,published=7000,source=7000,target=7000/udp python:3.10-slim python /here/udpserver.py

so I might be able to link all the ports together , which means back to a DDS question : I will need containers to be able to run on non-default ports like 7414, 7416. when I tried that before from XML they lost all comms. Do I need to do anything other than this, plus mapping the ports ?

# in the <builtin> bit

                    <metatrafficUnicastLocatorList>
                        <locator>
                            <udpv4>
                                <address>172.18.51.66</address>
                                <port>7414</port>
                            </udpv4>
                        </locator>
                    </metatrafficUnicastLocatorList>

# outside the <builtin> bit but inside the <rtps> bit

                   <defaultUnicastLocatorList>
                    <locator>
                        <udpv4>
                            <address>172.18.51.66</address>
                            <port>7415</port>
                        </udpv4>
                    </locator>
                </defaultUnicastLocatorList>  

[Oilecaker]

I am encountering a similar issue. I currently have an experimental scenario where I need to perform LAN communication tests between two machines using FastDDS. The runtime environment needs to be a Linux system under a Windows environment. For the first solution, I used WSL2, and for the second solution, I used the Windows version of Docker with an Ubuntu image. However, in both solutions, I was unable to establish LAN communication between the two hosts using FastDDS.
Check the ip route routing rules and the network interface details with ifconfig.

WSL2 uses a mirrored network but still cannot complete the communication. However, on the subscriber side, I can capture data packets on port 7400 using tcpdump -i eth0 udp port 7400 or udp portrange 7410-7420 -w fastdds_capture.pcap.
but I still get unmatched.