Asymmetric Routing on Linux: Why rp_filter Drops Your Packets (and the Policy Routing Fix)
A Linux box with two interfaces that answers on one network but ignores the other is one of those faults that looks like a firewall problem and is not. The packets arrive, tcpdump sees them, and the application never does. The culprit is usually rp_filter, the kernel’s reverse path filter.
I reproduced the whole thing on my Debian 13 test VM with network namespaces, so every output below is real.
The setup
A client c and a dual-homed host h are joined by two links. The client owns a “remote” address, 172.16.0.1, which it can send from over either link. The host’s route back to 172.16.0.1 only goes via link 2.
sudo ip netns add c
sudo ip netns add h
sudo ip link add c1 type veth peer name h1
sudo ip link add c2 type veth peer name h2
sudo ip link set c1 netns c; sudo ip link set c2 netns c
sudo ip link set h1 netns h; sudo ip link set h2 netns h
sudo ip -n c addr add 10.1.0.1/24 dev c1
sudo ip -n c addr add 10.2.0.1/24 dev c2
sudo ip -n c addr add 172.16.0.1/32 dev lo
sudo ip -n h addr add 10.1.0.2/24 dev h1
sudo ip -n h addr add 10.2.0.2/24 dev h2
for l in "c c1" "c c2" "c lo" "h h1" "h h2" "h lo"; do set -- $l; sudo ip -n $1 link set $2 up; done
sudo ip -n h route add 172.16.0.1/32 via 10.2.0.1 dev h2
The client pings the host’s link 1 address, sourced from 172.16.0.1. So the request arrives on h1, but the host’s best route back to 172.16.0.1 is out of h2. That mismatch is asymmetric routing.
What Debian ships
$ sysctl net.ipv4.conf.all.rp_filter net.ipv4.conf.default.rp_filter
net.ipv4.conf.all.rp_filter = 0
net.ipv4.conf.default.rp_filter = 2
On Debian 13, all is 0 and default is 2, so every new interface starts in loose mode. The effective value for an interface is the larger of all and the per-interface setting. The three modes are:
| Value | Mode | Rule |
|---|---|---|
| 0 | off | no check |
| 1 | strict | the best route back to the source must be via the interface the packet arrived on |
| 2 | loose | the source must be reachable via any interface |
Inside a fresh namespace, h1 and h2 also came up with 2. Many guides still say Linux defaults to strict. On this install it does not, but plenty of appliances, hardening guides and CIS-style profiles turn it to 1, so it is worth knowing both outcomes.
Strict mode: the packet vanishes
sudo ip netns exec h sysctl -w net.ipv4.conf.all.rp_filter=1 \
net.ipv4.conf.h1.rp_filter=1 net.ipv4.conf.h2.rp_filter=1 \
net.ipv4.conf.all.log_martians=1
sudo ip netns exec c ping -c 3 -W 1 -I 172.16.0.1 10.1.0.2
3 packets transmitted, 0 received, 100% packet loss, time 2047ms
tcpdump on h1 shows the requests arriving:
IP 172.16.0.1 > 10.1.0.2: ICMP echo request, id 55742, seq 1, length 64
IP 172.16.0.1 > 10.1.0.2: ICMP echo request, id 55742, seq 2, length 64
tcpdump sees packets before the IP stack makes its decision, which is why “I can see it in the capture” proves nothing about delivery. There is no reply, and nothing in the firewall counters. Two places do record it. The SNMP-style counter:
$ sudo ip netns exec h nstat -az | grep -i ReversePath
TcpExtIPReversePathFilter 2 0.0
and the kernel log, once log_martians is on:
IPv4: martian source 10.1.0.2 from 172.16.0.1, on dev h1
The wording is confusing. “Martian source” here does not mean the address is bogus. The message prints the destination first, then the source: a packet to 10.1.0.2 from 172.16.0.1 failed the reverse path check on h1. If you ever see this line, check the routing table before you check the firewall.
log_martians can be noisy on a busy box, so turn it on to diagnose and off again afterwards.
Loose mode: it works, but the reply goes the wrong way
sudo ip netns exec h sysctl -w net.ipv4.conf.all.rp_filter=2 \
net.ipv4.conf.h1.rp_filter=2 net.ipv4.conf.h2.rp_filter=2
sudo ip netns exec c ping -c 3 -W 1 -I 172.16.0.1 10.1.0.2
64 bytes from 10.1.0.2: icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 10.1.0.2: icmp_seq=2 ttl=64 time=0.088 ms
64 bytes from 10.1.0.2: icmp_seq=3 ttl=64 time=0.085 ms
3 packets transmitted, 3 received, 0% packet loss
Loose mode accepts the packet because 172.16.0.1 is reachable via some interface. But look at where the reply travels. Capturing on the client’s link 2:
IP 10.1.0.2 > 172.16.0.1: ICMP echo reply, id 55765, seq 1, length 64
The request came in on link 1 and the reply left on link 2, with a source address that belongs to link 1. That is asymmetric routing in the flesh. It works here because the client’s own rp_filter is 0. When I set strict mode on the client as well, the lab went back to 100% loss and the counter incremented on the client’s side. A stateful firewall or NAT device sitting on one of those paths would see half a conversation and drop it too.
So loose mode stops the drops but does not fix the asymmetry. It hides it.
The real fix: policy routing
The reply to a packet that came in on h1 should go out of h1. Linux can do that with a second routing table and a rule that selects it by source address:
sudo ip -n h route add 172.16.0.1/32 via 10.1.0.1 dev h1 table 101
sudo ip -n h rule add from 10.1.0.2 lookup 101 priority 100
$ sudo ip -n h rule show
0: from all lookup local
100: from 10.1.0.2 lookup 101
32766: from all lookup main
32767: from all lookup default
Then route get shows the host picking a path by source address:
$ sudo ip -n h route get 172.16.0.1 from 10.1.0.2
172.16.0.1 from 10.1.0.2 via 10.1.0.1 dev h1 table 101
$ sudo ip -n h route get 172.16.0.1 from 10.2.0.2
172.16.0.1 from 10.2.0.2 via 10.2.0.1 dev h2
With strict mode back on (rp_filter=1) the ping works and the symmetry is restored. The capture on the client’s link 1 now shows both directions:
IP 172.16.0.1 > 10.1.0.2: ICMP echo request, id 55787, seq 1, length 64
IP 10.1.0.2 > 172.16.0.1: ICMP echo reply, id 55787, seq 1, length 64
Strict mode now passes because the best route back to 172.16.0.1 for a packet arriving on h1 is genuinely via h1. Note that the kernel evaluates the reverse path with the incoming interface in mind, which is why the rule fixes the filter and the replies at the same time.
To make it survive a reboot, put the rule and route into your network configuration. With systemd-networkd that is a [Route] with Table=101 and a [RoutingPolicyRule] with From=10.1.0.2 and Table=101. With ifupdown, post-up ip rule add ... lines work.
Which setting should you use?
- Single-homed hosts: leave Debian’s default alone.
- Multi-homed hosts with proper policy routing: strict (1) is the safest, and the one most security baselines ask for.
- Multi-homed hosts where you cannot make routing symmetric, such as ECMP or traffic arriving from a provider that load balances: loose (2). Do not use 0 unless you have another anti-spoofing control.
Debug checklist
sudo sysctl net.ipv4.conf.all.rp_filter net.ipv4.conf.<if>.rp_filter: remember the effective value is the larger of the two.nstat -az | grep ReversePath: is the counter increasing while you test?sudo sysctl -w net.ipv4.conf.all.log_martians=1, thendmesg | tail.ip route get <source> from <local-ip> iif <interface>: which way would the kernel go back?ip rule show: is there a policy that explains the answer?
The counter and the log line turn an invisible drop into a two minute fix.