Container Networking Deep Dive Part 5: What CNI Actually Is

Every post so far in this series has been Docker networking things Docker itself does: docker0, the driver selection, the overlay network. Kubernetes does not use any of it. A Kubernetes node runs a container runtime (containerd, in most current clusters) to start containers, and a completely separate component, a CNI plugin, to wire up their networking. This post is about why that split exists and what a CNI plugin’s job actually is, not a Kubernetes networking deep-dive. The two things Kubernetes needs from a network, Services and Ingress, are a different problem from what CNI solves and are out of scope here on purpose.

The problem Docker’s model doesn’t solve

Docker’s bridge model, covered in Part 2, gives every container an IP from a private range local to that host, reachable from outside only through NAT and published ports. That’s fine when a container is a self-contained unit calling out and being called via a handful of known ports. It breaks down for the specific thing Kubernetes wants: every pod gets a real, routable IP, and every pod can reach every other pod’s IP directly, cluster-wide, with no NAT and no port publishing, regardless of which node either one lands on.

Docker’s own default networking has no notion of “every container’s IP is directly reachable from a container on a different host without going through a published port.” Its answer to cross-host reachability, covered in Part 4, is an overlay network you deliberately create, a specific network, not the default behavior of every container in the cluster. Kubernetes needs the cross-host, no-NAT reachability to be the default and only behavior for every pod, unconditionally. That’s a different enough requirement that Kubernetes didn’t build its own bridge-driver equivalent. It defined an interface and let other people implement it.

The interface: two verbs

CNI (Container Network Interface) is a specification, not a piece of software. It defines a plugin as an executable binary that gets invoked by the container runtime with exactly one of two commands, plus a JSON configuration blob on stdin describing what network to attach the container to:

ADD: attach a container’s network namespace to the network. The plugin is handed the namespace path and a network name, and is responsible for creating whatever interface, IP assignment, and routing the network implementation needs, a veth pair to a bridge, an entry in a distributed routing table, a VXLAN tunnel endpoint, whatever the specific plugin does internally. What comes back on stdout is a JSON result: the IP assigned, the interface name, the routes installed.

DEL: tear down whatever ADD created for that specific container, cleanly, on pod deletion.

That’s the entire lifecycle. No UPDATE, no live reconfiguration verb. A running pod’s network attachment is immutable from CNI’s point of view; changing it means deleting and recreating the pod.

# Roughly what kubelet does when a pod starts, simplified:
CNI_COMMAND=ADD \
CNI_CONTAINERID=abc123 \
CNI_NETNS=/var/run/netns/cni-abc123 \
CNI_IFNAME=eth0 \
/opt/cni/bin/<plugin-binary> < /etc/cni/net.d/10-plugin.conflist

The plugin binary reads the config from stdin, does whatever work its implementation requires, and writes a result back to stdout as JSON. kubelet never has an opinion about how the network gets built, bridge, VXLAN, BGP-distributed routes, eBPF maps, only that ADD returns a working interface with an IP, and DEL removes it again.

Why one-IP-per-pod forced this split

The Kubernetes networking model has one flat rule underneath everything else: every pod gets its own IP, and that IP is reachable from every other pod in the cluster without NAT. This is a deliberate, explicit requirement in the Kubernetes networking model, not an emergent property. It means:

• No two pods on the same node can share an IP, the same way two Docker containers on the same bridge never share one. This part is identical to what Part 2 already covers.
• A pod on node A must be reachable by its own IP from a pod on node B, with no port publishing, no NAT, no translation of any kind. This is the part Docker’s bridge model has no equivalent for at all, and the part Docker’s overlay driver only provides when you opt into it explicitly.

Achieving the second point cluster-wide, automatically, for every pod, requires some mechanism to make a pod’s private IP routable outside its own node: an overlay tunnel (Part 4’s VXLAN, applied automatically instead of opt-in), a real routing protocol distributing pod routes between nodes, or an eBPF-based approach that bypasses conventional routing entirely. Which mechanism to use is a genuine architectural choice with real tradeoffs, covered driver by driver in the next post, and that choice is exactly what CNI plugins exist to make different ones of, behind one stable interface kubelet doesn’t need to know the details of.

What actually happens when a pod starts

Zoom into a single pod’s ADD call. The runtime has already created the pod’s network namespace before invoking the plugin. This part is identical to docker run creating a namespace before attaching it to a bridge. The plugin then, at minimum:

1. Creates a veth pair, one end into the pod’s namespace as eth0, the other end staying on the host (or gets attached to whatever mechanism the plugin uses instead of a bridge)
2. Assigns an IP to the pod-side end, from whatever address range that node has been allocated out of the cluster’s overall pod CIDR
3. Installs whatever routing or forwarding rule makes that IP reachable: a bridge attachment for local traffic, plus whatever cross-node mechanism (VXLAN tunnel entry, BGP route announcement, eBPF map entry) the plugin uses for pod-to-pod traffic that has to leave the node
4. Returns the assigned IP and interface name as JSON, which kubelet records against the pod

Steps 1 and 2 are the exact operations from Part 1’s hand-built lab and Part 2’s Docker-automated version. Step 3 is the part that’s genuinely new, and the part where Flannel, Calico, and Cilium, covered in the next post, diverge from each other and from Docker’s overlay driver, each solving cross-node reachability with a different mechanism and a different set of tradeoffs.

What CNI deliberately doesn’t do

CNI has nothing to say about Kubernetes Services, kube-proxy’s load-balancing, or Ingress. Those are separate Kubernetes components layered on top of whatever IP-per-pod reachability CNI provides underneath. A CNI plugin’s job ends at “this pod has an IP and can reach other pods’ IPs.” What happens to traffic aimed at a Service’s virtual IP, and how that gets load-balanced to one of several backend pods, is a different subsystem, and stays out of scope for this series by design. This series is about the network CNI builds, not the workload-routing layer Kubernetes builds on top of it. Cilium’s kube-proxy replacement, covered in the eBPF-focused post later in this series, is the one place that boundary gets blurred, because Cilium’s eBPF datapath ends up doing both jobs in one program, worth calling out there specifically, not assumed here.

The next post takes three real CNI plugins, Flannel, Calico, and Cilium, and compares how each one actually implements the ADD verb’s cross-node reachability requirement, because “attach a veth pair and assign an IP” is identical across all three. The entire difference between them is in what happens next.