XG v18 MR-3 SDWAN Policy Routing

Do you have a SD-WAN Rule on SiteB? 

The important part to understand is: SD-WAN PBR will break with the network expectation of the last decades. You can "force" XG to route those packets to other interfaces.

If you enable "reply-packets", it will even route the packets, based on a existing connection, to another Interface and not to the interface, which the packets originated. 

For years, this setup was feared as asymmetric routing and bad practice. But starting with SD-WAN, you actually do not care, how those packets get to the destination, as long they arrive there. 

We don't have any migrated SDWAN rules. We were using were using straight IPSEC site-to-site tunnels. We are moving them to VTI for better failover control.

I will try this tonight, but I don't expect it it will make a difference. That is how it was before.

I did notice last night that when I switched HA to the passive device (failed over), traffic did work as expected for a time. I have to wonder if VTI/PBR works as expected in HA setups.

I have a question. Do you MASQ/NAT the traffic on one XG for the tunnel? 

Not intentionally. I haven't gone thru the NAT rules after the v18 upgrade though.

Site A - 192.168.x.0/24,  Site B - 192.168.y.0/24

Static route works... I Would expect a NAT/MASQ issue would show up there as wll.

Do you have a NAT Rule from Site A to Site B using SNAT? Anything to masq the traffic? 

No

You should get in touch with Sophos Support to get your conntracks etc. analysed. What you could do, try to disable firewall acceleration to check, if this will actually have an impact. 

Where is that?

console> system firewall-acceleration disable

I do not expect to fix your issue, but its worth a shot. 

So I logged into the 2 firewalls, enabled my SDW routing, and disabled my static routes. And it worked. I cleared the CONNTRACK on both FWs and it still worked. Odd, since I hadn't many any changes. Failed over HA on one side, everything failed. I couldn't get the SDW routing to work no matter what I did, failed back, reset the VTI tunnels, CONNTRACK, etc. Put the manual routes back in and everything works. I don't know what the issue is, but it doesn't work!!!!    Argh!

So I ended up speaking with an engineer (not support) and he was able to verify in CONNTRACK that the reply packets were not being tagged by the SDWAN policy. Then of course it started working and we couldn't get it to fail again. I moved over another remote site last night, and it failed immediately.  Same issue, SDWAN routes not working but static routes are. 

So I ended up speaking with an engineer (not support) and he was able to verify in CONNTRACK that the reply packets were not being tagged by the SDWAN policy. Then of course it started working and we couldn't get it to fail again. I moved over another remote site last night, and it failed immediately.  Same issue, SDWAN routes not working but static routes are. 

i have the same issue on mr4.

i have a default  sd-wan policy route with destination any protocol any for my internet with a more specific route also sd-wan for internal traffic and static route works, but policy route makes it use the public internet and never uses my private circuit nor my vpn path, even though both are up.

Did you ever figure this out?