How to route traffic between multiple logical routers aka Inter-LR Routing on a Palo Alto Networks Strata firewall? More precisely, inclusive route redistribution rather than a few static routes. –> Via iBGP through loopback interfaces. ✅ Let’s go:
Setup Notes
- The advanced routing engine (ARE) is in place 👍🏻, hence: “logical routers” (LR) rather than legacy “virtual routers” (VR).
- Each LR has a loopback interface and requires a static route of type “next-lr” to the loopback interface of the *other* LR. This is the internal-routing-path-trick. ;)
- A security policy must allow “bgp” between the zones of those loopback interfaces.
- Internal BGP (iBGP) is used for this setup. The “Next Hop: Self” property must be set between those peers to route all traffic through these loopback interfaces.
- All further security policies for normal data traffic are built upon the original source-/destination zones of the data interfaces. No special external zones nor additional policies are required! This is great and keeps it simple. From a security policy’s point of view, it behaves like one single routing process. (Note that this is different compared to inter-VSYS routing which requires special “External” zones and a policy in each VSYS.)
- Interestingly (consequently?), this internal hop is *not* visible via traceroute.
- If you have more than 2 LRs, you must set up a full mesh between all LRs and their loopback interfaces. Internal routing from LR1 via LR2 to LR3 is not supported.
- BGP can be used as a dynamic routing protocol for this inter-LR routing since BGP neighbouring can be done via static routes. This is not possible for OSPF. If you want to use OSPF for inter-LR routing, you have to use hardware interfaces rather than loopback interfaces. This won’t scale, obviously.
- Used equipment for this post: PA-440 with PAN-OS 11.2.0, “Advanced Routing” enabled; AS 64512
- The left LR, “default”, redistributes its connected and static routes to BGP.
- The right LR, “service-provider”, is connected to a Cisco router (2811 with IOS 15.1(4)M12a) via eBGP, AS 64513, and automatically sends these routes through the BGP process to the left LR. No explicit redistribution is required.
- I’m following the best practice for dual-stack BGP scenarios, that is: IPv6-neighbours for IPv6-routes; and IPv4-neighbours for IPv4-routes.
A picture is worth a thousand words:
Screenshot Guide
Good luck. 😂 I’m not showing everything here. Everything that is done within the “default” LR is mirrored in the “service-provider” LR as well, though not explicitly screenshotted.
Show Notes
“More Runtime Stats” from the “service-provider” LR, right-hand side:
Tracerouting from a PC behind the Cisco router (1st hop) through BOTH LRs on the Palo (2nd hop <- yes, just one!) to a Linux machine (3rd line) behind the “default” LR on the Palo:
|
1 2 3 4 5 6 7 8 9 10 |
weberjoh@nb15-lx:~$ sudo traceroute -I nuc.weberlab.de traceroute to nuc.weberlab.de (2a00:6020:ad0b:8300::c01:22), 30 hops max, 80 byte packets 1 2a00:6020:ad0b:8310::1 (2a00:6020:ad0b:8310::1) 1.773 ms 2.353 ms 2.741 ms 2 2a00:6020:ad0b:8370::1 (2a00:6020:ad0b:8370::1) 1.011 ms 1.006 ms 1.002 ms 3 pi.hole (2a00:6020:ad0b:8300::c01:22) 1.597 ms 1.592 ms 1.587 ms weberjoh@nb15-lx:~$ sudo traceroute -I 192.168.7.53 traceroute to 192.168.7.53 (192.168.7.53), 30 hops max, 60 byte packets 1 _gateway (10.10.10.1) 1.762 ms 2.376 ms 2.960 ms 2 10.0.70.1 (10.0.70.1) 0.843 ms 0.839 ms 0.835 ms 3 pi.hole (192.168.7.53) 1.665 ms 1.663 ms 1.708 ms |
Some CLI commands for further troubleshooting. Note the possibility of capturing on individual routing protocols:
|
1 2 3 4 5 6 |
show advanced-routing bgp ? show advanced-routing route debug advanced-routing bgp ? debug advanced-routing pcap bgp on logicalrouter <name-of-the-LR> debug advanced-routing pcap bgp off scp export debug-pcap from bgp-vr-2.pcap to username@host:path |
For example, this is a small capture I exported in my lab: (Only KEEPALIVEs here, sorry ;))
That’s it. ✅
PS: I’m not a BGP pro, though. If you encounter any inconsistencies, please write a comment down below! Thanks.
Soli Deo Gloria!
How would you accomplish dynamic routing between two logical routers, using inter-vsys going through “External” zones? So that it never leaves the firewall.
Hey Ed. That is: You want to route traffic between different *VSYS*s (each with its own VR/LR) rather than within a single VSYS with multiple VR/LRs, right?
(By the way: You can use multiple VSYSs with just one VR/LR, just in case.)
Anyway, I would try the same iBGP approach as shown here, that is: loopback interfaces in each VR/LR, static routes to these loopback through “next VR”, and of course, appropriate security policies in each VSYS such as shown here: https://knowledgebase.paloaltonetworks.com/KCSArticleDetail?id=kA10g000000ClSVCA0. The last step makes the difference, since you’re within multiple VSYSs now.
That’s correct. Different VSYS’s, each with its own LR. Communication through “External” zones. Thank you for reply and I will try it.