0

I am attempting to run multiple (we'll say 2) instances of software in docker. This software is proprietary and 3rd party, so modifying it is not an option, and neither is using docker exec or attach to make changes from inside the containers.

The software in question launches a web GUI at port 80. It also receives traffic on port 50000. The traffic we expect to send to port 50000 is extremely latency sensitive and at 10G.

The host has multiple interfaces:

bond0 172.16.2.10 
bond0:0 172.16.2.20
bond0:1 172.16.2.21
em3 172.16.3.10
em4 172.16.4.10

Bond0 is the host's primary connection to the LAN. The aliases 0 and 1 are there because we're running 2 copies of the software, and for simplicity it made sense if they each have their own IP. This might be the wrong approach, so I can remove these aliases if needed.

EM3 and 4 are fiber connections directly to an appliance that will be sending the 10G latency sensitive traffic on port 50000. EM3 is designated for the first instance and EM4 for the second. These assignments are physical and cannot be changed, but if I need to change the configuration of these connections that is fine.

Originally I used the default bridge network for all of this, mapping specific ports and IPs to each container. This worked excellently for connecting at first, the web UIs were accessible and the fiber connections were up. However, I ran into issues soon after: Docker's bridge network driver is limited to 1500 MTU, and the traffic headed over EM3 and EM4 requires an MTU of 9000. Attempts to set the bridge network at MTU 9000 using daemon.json or spinning up a new bridge network have proven unsuccessful.

I really would like to use net=host for these containers, but port conflicts would immediately occur and not all of the ports that the software requires are customizable. There is no way in the software to bind to specific network interfaces. I also explored macvlan_mode=passthru, setting up a macvlan for each network interface and assigning the relevant ones to containers, but I am not sure if this is the right path.

Anyone have any ideas?

Improve this question
2
  • This article suggests how to change the MTU used in Docker containers. I am not able to test this myself at the moment.
    – larsks
    Commented Mar 12 at 1:23
  • Yes, I've reviewed that article in my search for a soluton. It works fine for switching between MTUs that docker's bridge driver supports (anything 1500 and below), but is not effective for enabling jumbo frames.
    – user23564615
    Commented Mar 12 at 1:56

1 Answer 1

Reset to default
0

I gave up on finding a particularly clean solution and enabled host networking. In order to prevent port conflicts, I set up iptables forwarding and masquerade rules that switch the ports around behind the scenes depending on what IP address and port traffic is sent to or from.

This works for accessing the software GUI (it defaults to typical HTTPS port but can be reassigned with an environment variable) but not for all other features. Thankfully I can update the latency sensitive traffic ports as well. I wish there was a better way, but this does resolve my MTU issue, even if it is messy and imperfect.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.