This goes out to anyone who uses more than one Site-to-Site VPN tunnel between two locations that are secured by firewalls from Palo Alto Networks. Using two (or even more) VPN tunnels, you need an automatic way to failover the traffic flow from one VPN to the other in case of failures. Here’s how to accomplish that requirement:

Pre-Notes

• You need 2x Site-to-Site VPNs between two firewalls. There may be 2x different ISPs on both sides, but at least on one side. Each ISP connection needs its own logical/virtual router to have its own default route.
• All tunnel interfaces on both sides must use IP addresses.
• All tunnel interfaces can remain in the same and single LR/VR. That is: Even though different LR/VRs are used for the ISP connections and the tunnel establishment, the final tunnel interfaces can be in the same LR/VR.
• You need 2x static routes to the same destination through those 2x different tunnel interfaces. The “primary” VPN should use a more preferred metric (i.e., lower value). This must be consistent on both sides! If not, you’ll run into asymmetric routing problems as both VPN tunnels will be up most of the time.
• Using a monitor profile with an action of “Fail Over” within the IPsec tunnel configuration takes a non-working VPN tunnel interface down. –> The (formerly preferred) static route will be kicked out of the forwarding table. ✅
• Of course, you could use a dynamic routing protocol such as OSPF or BGP to accomplish the same, while having a more complex overhead.

Some notes about my lab:

• My firewall “pa-home” on the left is connected to two different ISPs. Both offer dynamic IPv4 addresses only, hence I’m using a peer address type “dynamic” from the other firewall. Different local IDs are used to get those two tunnels authenticated. Terminates various L3 subnets; the summary route to this location simply sets 192.168.0.0/16 as the destination.
• (1x DS-Lite with CGNAT over fibre from “Deutsche Glasfaser” (DG), using DHCP and DHCPv6-PD, 1x Dual Stack over DSL from “Deutsche Telekom” (DTAG), using PPPoE and PPPoEv6. Though these details are not of interest here. :))
• Primary VPN through DTAG, tunnel.12, 172.16.12.0/30, routing metric of 20
• Secondary VPN through DG, tunnel.11, 172.16.11.0/30, routing metric of 50
• My firewall “pa-stg” on the right is behind a BGP-routed prefix (connected to 2x ISPs), with the advantage that it only has a single public IPv4 address, though high available due to BGP. Here, only one LR is used at all. Terminates a single L3 subnet: 192.168.4.0/24.
• pa-home is PA-440 with PAN-OS 11.2.9, Advanced Routing enabled
• pa-stg is a PA-460 with PAN-OS 11.2.4-h7, Advanced Routing enabled as well

The two routes on the pa-stg firewall are configured like this. No specials:

What if: Usage without any Monitoring

Just in case you’re interested: Without any monitoring profile set on those VPN tunnels, both routes will be present in the routing table (RIB), while the one with the better metric will be in the forwarding table (FIB). This is true even if one or both VPN tunnels are down!

Using a Monitor Profile with “Fail Over”

It’s fairly easy now: Just configure the tunnel monitor within each IPsec tunnel with a profile that has the action of “Fail Over” set, pinging the tunnel interface of the other side:

If one ISP connection or VPN tunnel fails, you’ll see the following “tunnel-status-down” system log, while the tunnel interface status in the IPsec tunnels section goes down (!), ending in only one route in the RIB as well as in the FIB:

This is how the user experience appeared. Also note the difference in the RTT, decreasing from 15 ms to 9-10 ms, depending on the ISP’s routing policies.

If the primary tunnel is up again, you’ll see a “tunnel-status-up” in the system log, while the primary route will be in the RIB/FIB again. Of course, on both sides of the VPN-tunnel simultaneously, since otherwise you would have asymmetric routing. Again, there are some pings lost, while the RTT (in my scenario) increased from 10 ms to 15 ms:

Final note: While I used the “Tunnel Monitor” section within the IPsec tunnels for this failover, you could also use the “Path Monitoring” within the static routes. It functions almost the same since it pings the other side of the VPN tunnel and only adds the routes in case of a working connection. However, to my mind, it’s more sound to get the tunnel interface down rather than simply deleting the route. I want this failover decision to be as close as possible to the IPsec tunnel configuration. But that’s an architectural choice.

Anyway. Happy failovering. ;)

Soli Deo Gloria!

Photo by Caleb Jones on Unsplash.