Firewall Traffic gets routed in full-tunnel IPsec VPN

Hi Tobias Kuhnert : Thank you for reaching out to Sophos community team. What is the route precedence set over XG as of now? 

The default route precedence in 18.0 is set to static, SD-WAN policy routes, and VPN. Can you please check the below KBA for reference steps and set the route precedence to it's default V18 settings, if it not set.
 
https://support.sophos.com/support/s/article/KB-000037964?language=en_US

https://docs.sophos.com/nsg/sophos-firewall/18.0/Help/en-us/webhelp/onlinehelp/nsg/sfos/learningContents/ConfiguringRoutePrecedence.html

Note: You may test it in odd hours for safer side to avoid any outage or issue.

console> system route_precedence show
Routing Precedence:
1. Static routes
2. SD-WAN policy routes
3. VPN routes

"default dev ipsec0 table 220 scope link src 10.64.0.1 <-maybe this is the issue?"

that is the cause of your issue

Edit: but this is only for table 220 - the IPSec Networks (ip route show table 220 (don't post the result))

SG230_WP02_SFOS 18.0.4 MR-4# ip route show table wanlink1
default via 172.16.64.145 dev PortE1 proto static src 172.16.64.148
prohibit default proto static metric 1

re-saving the interface configuration had no effect. traffic is still routed into the tunnel.

i think there is nothing sensitive in it:
SG230_WP02_SFOS 18.0.4 MR-4# ip route show table 220
default dev ipsec0 scope link src 10.64.0.1

any thoughts what to check or try next?

220 should print out all remote networks of your IPSec tunnel. But maybe in your case you do not need access from there to remote networks (e.g. HQ networks), only the other way around. So it may be empty.

The only difference in your config I could find is the routin precedence:

Default routing Precedence:
1.  SD-WAN policy routes
2.  VPN routes
3.  Static routes

yours:
Routing Precedence:
1. Static routes
2. SD-WAN policy routes
3. VPN routes

We have no SD-WANs

We have VPN routes (but only to the 220 networks, no WAN routes)

-> system ipsec_route show

You say your static routes are empty. And so are ours.

It's worth a try to change that precedence.

Maybe LuCar Toni ca have a review on your NAT/SNAT config?

and please be sure that you don't have bad SNAT here:

console> show advanced-firewall

...

NAT policy for system originated traffic
---------------------
Destination Network Destination Netmask Interface SNAT IP

interesting stuff here

I am not sure right now, which will be applied first. If you change the Traffic via a NAT Rule, if the traffic will hit the IPsec0 Interface or if the routing decision will be applied after the NAT. I assume its the NAT part. Having said, this is rather a problematic construct, as the firewall is doing what you tell the firewall to do. It NATs the traffic to ANY into the tunnel. 

I would assume, you should think about the setup in the first place, if remote subnet should be ANY and cannot be more specific? 

one thing to point out: We do not apply NAT to any traffic in the tunnel.

we have a local subnet 10.64.0.0/21 and the remote subnet is any or 0.0.0.0/0 - no NAT applied

we need the full tunnel because web-security is currently applied in the HQ.

the only NAT rule i have is from LAN to WAN zone for direct internet access if the tunnel fails.

console> show advanced-firewall
NAT policy for system originated traffic
---------------------
Destination Network Destination Netmask Interface SNAT IP

... list is empty

Somehow the XG uses 0.1 as a outgoing IP. This is configured by somebody. Please review the NAT page and check all rules. 

please see the NAT rules below. i can't see anyhting wrong here.

you may ignore rule 1 - this is a workaround to have the DNS server function working for 1 client - does not make difference if i delete/disable it.

please see the NAT rules below. i can't see anyhting wrong here.

you may ignore rule 1 - this is a workaround to have the DNS server function working for 1 client - does not make difference if i delete/disable it.

Do you use NAT on the IPSec setup? The Checkbox? 

PS: What is the issue, if you wants to use the IPsec Tunnel for everything? I mean, if Security is done on the peer site, you can also give the DNS etc. to them? 

not checked

sure i could allow the XG traffic on the peer side - that would be the easy way.

but i don't get why an update of the firewall changes the routing in a way that all my security policies on the peer side need to be edited.

and i would expect the XG firewall system to be configurable in the way i want it... and since is was working before in that way, i would rather find the cause of the issue and solve it

Basically there were couple fixes of certain behaviors and fixed issues. Some of them caused some weird behavior. And i assume, you saw this in the old version. Now it is doing exactly what the firewall should do. Send everything to the peer end. 

To change this, its not that easy, as its system generated traffic. System generated traffic is a special use case in Firewall business. The firewall can apply Sd-WAN PBR rules to it. But as you use ANY, it would overwrite your VPN rule and would destroy your deployment. 

I cannot think of a valid configuration yet for your setup. The use case is not valid right now, because why would you want to use the local breakout for DNS and request everything to the peer end? 

i think the problem is more that the firewall is not using its WAN interface IP as source. I don't know what the source was in the previous version tbh.

Otherwise i guess i wouldn't be able to connect to the WAN interface for management (i am doing this right now all the time) if default routing for the whole system would really be the ipsec route.

any chance we can change the source interface for the "system-generated" traffic?

i already tried with the "sys-nat-traffic" rules for explicitly the 8.8.8.8 as destination - no luck
i also tried PBR rules - no luck (not sure if the configuration was correct:

You cannot manipulate the system generated traffic that easy. See: https://support.sophos.com/support/s/article/KB-000035607?language=en_US Thats the only switch to manipulate this. 

i tried this - not working

maybe this is somehting which Sophos should figure out or fix. seems really like an issue in combination with full tunnel VPNs...

To come to an end i'll reconfigure the peer side to allow the XG traffic via the tunnel as a workaround for now.

thanks for the support LuCar Toni and LHerzog