Container Networking Deep Dive Part 1: From Namespaces to a Container Network by Hand

Docker gets described as a networking technology, but it does not introduce any new networking primitive. Every container network Docker builds is made of three things Linux already had: network namespaces, veth pairs, and a bridge. The namespaces post in this series used those same primitives to wire up a multi-router BGP and OSPF lab. This post uses them to wire up something that looks a lot more like a container network, by hand, with no Docker installed, so the next post’s docker0 bridge and iptables rules read as recognizable rather than magic.

The goal by the end of this post: two isolated “containers” (they are network namespaces, but the pattern is identical) on one host, connected through a bridge, each with its own IP, and able to reach the outside world through NAT written with plain iptables. That is a bridge network, and that is all a bridge network is.

The topology

                         host namespace
                    ┌─────────────────────┐
                    │   br-containers      │
                    │   172.20.0.1/24      │
                    └──┬────────────────┬──┘
                        │                │
                  veth-app-a       veth-app-b
                        │                │
              ┌─────────┴──────┐ ┌───────┴─────────┐
              │  netns app-a   │ │  netns app-b     │
              │  eth0          │ │  eth0            │
              │  172.20.0.10/24│ │  172.20.0.11/24  │
              └────────────────┘ └──────────────────┘

app-a and app-b are the containers. br-containers is the bridge Docker calls docker0 when it builds the default network. veth-app-a and veth-app-b are the host-side ends of the veth pairs whose other ends live inside each namespace as eth0, and every container’s eth0 you have ever seen with docker exec <container> ip addr is one end of a veth pair exactly like this.

Building it

Create the two namespaces:

sudo ip netns add app-a
sudo ip netns add app-b

Create the bridge and bring it up:

sudo ip link add br-containers type bridge
sudo ip addr add 172.20.0.1/24 dev br-containers
sudo ip link set br-containers up

Wire app-a to the bridge. The veth pair is created in the host namespace, one end renamed and moved into app-a, the other end attached to the bridge as a bridge port:

sudo ip link add veth-app-a type veth peer name eth0 netns app-a
sudo ip link set veth-app-a master br-containers
sudo ip link set veth-app-a up

sudo ip netns exec app-a ip addr add 172.20.0.10/24 dev eth0
sudo ip netns exec app-a ip link set eth0 up
sudo ip netns exec app-a ip link set lo up
sudo ip netns exec app-a ip route add default via 172.20.0.1

Repeat for app-b:

sudo ip link add veth-app-b type veth peer name eth0 netns app-b
sudo ip link set veth-app-b master br-containers
sudo ip link set veth-app-b up

sudo ip netns exec app-b ip addr add 172.20.0.11/24 dev eth0
sudo ip netns exec app-b ip link set eth0 up
sudo ip netns exec app-b ip link set lo up
sudo ip netns exec app-b ip route add default via 172.20.0.1

At this point app-a and app-b can already reach each other, through the bridge, with no routing involved. A bridge forwards at layer 2 based on MAC address, the same as a physical switch:

sudo ip netns exec app-a ping -c 2 172.20.0.11

This is container-to-container traffic on the same Docker bridge network. It never leaves the host, never touches iptables, and never involves the default route. Docker’s default bridge behaves identically for two containers on the same network.

Reaching the outside world

app-a has a default route pointed at the bridge, but the bridge itself has no path to the internet unless the host forwards for it. Two things are missing: IP forwarding, and a NAT rule so return traffic from the internet knows where to come back to (the host’s real address, not the private 172.20.0.0/24 range, which is not routable outside this machine).

Enable forwarding:

sudo sysctl -w net.ipv4.ip_forward=1

Add the NAT rule. This is the entire content of what Docker calls the bridge network’s “masquerade rule,” one line:

sudo iptables -t nat -A POSTROUTING -s 172.20.0.0/24 ! -o br-containers -j MASQUERADE

Read literally: for any packet sourced from the 172.20.0.0/24 range that is leaving via any interface other than the bridge itself, rewrite the source address to the host’s own outbound address before it leaves. The kernel tracks the mapping in conntrack so return traffic gets un-rewritten and delivered back to 172.20.0.10 correctly.

From inside app-a:

sudo ip netns exec app-a ping -c 2 8.8.8.8

That now works, assuming the host itself has a route to the internet on its own uplink interface. Nothing about DNS resolution has been set up yet. app-a has no /etc/resolv.conf worth trusting inside the namespace, so a raw IP ping is the right test here, not a hostname.

Publishing a port

The last piece is the direction Docker calls “publishing a port”: letting something outside the host reach a service running inside app-a. Say app-a is running a listener on TCP/8080. To expose that on the host’s own port 8080:

sudo iptables -t nat -A PREROUTING -p tcp --dport 8080 \
  -j DNAT --to-destination 172.20.0.10:8080

sudo iptables -A FORWARD -p tcp -d 172.20.0.10 --dport 8080 -j ACCEPT

The first rule rewrites the destination of anything arriving at the host on TCP/8080 to 172.20.0.10:8080, before routing decides anything. The second rule permits the forwarded packet through the FORWARD chain, which defaults to DROP on most distributions once ip_forward is enabled with any restrictive baseline policy. This pair of rules is exactly what docker run -p 8080:8080 ... generates for you, with a container-ID-derived chain name instead of the default PREROUTING/FORWARD chains and a bit more bookkeeping so Docker can clean the rules up again when the container stops.

What Docker actually automates

Nothing in this post required a container runtime. Every piece, namespace creation, veth pair creation, bridge attachment, address assignment, the MASQUERADE rule, the DNAT/FORWARD pair for port publishing, is plain ip and iptables. What docker run adds is:

  • Doing all of the above automatically, per container, in the right order, without typos
  • Tracking which rules belong to which container so docker stop can remove exactly the right ones
  • An embedded DNS resolver so containers can reach each other by name instead of by the IP you assigned by hand
  • A default bridge network created once (docker0) that every new container joins unless told otherwise, instead of one bridge per manual lab

The next post in this series builds the same topology with docker run instead of ip netns, and looks at exactly what iptables-save shows once Docker is managing the rules instead of a shell script. If any of the commands here look unfamiliar on their own, the namespaces and veth pairs post covers the underlying primitives in more depth, from the same starting point of a bare Linux host with no orchestration layer at all.

Cleaning up

sudo ip netns del app-a
sudo ip netns del app-b
sudo ip link del br-containers
sudo iptables -t nat -D POSTROUTING -s 172.20.0.0/24 ! -o br-containers -j MASQUERADE
sudo iptables -t nat -D PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 172.20.0.10:8080
sudo iptables -D FORWARD -p tcp -d 172.20.0.10 --dport 8080 -j ACCEPT

Deleting a namespace tears down anything inside it, including its end of every veth pair, the same cleanup behavior noted in the namespaces post. The bridge and the iptables rules live in the host namespace, so they need removing explicitly.