News about IPsec speed issues and XG105

Like lots of people here, we have IPsec speed/bandwidth issues with SFOS and the "small" hardware appliances sold by Sophos.
Especially "old" ones (a couple of years old), the ones going EOL next july.

Here's a post about our last tests about this.

Main office is setup with a SFV6C8 virtual appliance.
That's 6 cores of some E5-2695 and 8 GB of RAM.
Currently running SFOS 17.5 (will be update to 18 later).
Gigabit link to the internet, 10 Gbps vNIC, hosting node connected with 10 Gbps to datacenter fabric.

Remote office is setup with an XG105 rev2.
That's an Atom E3826 with 2 GB of RAM.
Last SFOS (supported by this platform) is 17.5 and won't get SFOS18 (because RAM).
Network link is a dedicated guaranteed fiber of 200 Mbps to datacenter (same fabric than the nodes hosting the virtual appliance).

Remote office XG105 does nothing (no proxy, no filtering, nothing) but an IPsec tunnel to main office.
All trafic goes through this tunnel.

Speedtest from the main office gives us about 800 Mbps.
The main office virtual appliance is not overloaded.


Now the (bad) numbers.

If the tunnel is down (and NAT activated in the remote office), speedtest maxes out the available bandwidth.
That's 200 Mbps up and down from the remote office to the internet.

If the tunnel is up (remote office -> tunnel -> main office/datacentre -> internet), max speed is 70 Mbps.

The numbers are the same for all the tests branches offices.
The ones with 100 Mbps link maxes out the link with NAT and don't go above 70 Mbps with IPSec.
Same for a home office with FTTH, 400 Mbps with NAT and 70 Mbps with IPsec.

Several ciphers were tried (AES126 and AES128), didn't help.

There are several thread here (and several opened cases) about this.
The problem exists and is known.
Current suggested solution is "upgrade your remote office hardware, the one you have is not beefy enough".


And now, today's first test.

We took one spare XG105 and installed opnsense on it.
And we setup an IPsec tunnel to the main office/datacentre, exactly the same way than with SFOS.
Speedtest using the tunnel is about 90-100 Mbps.
Ok.
So it's not JUST the hardware.

We tried to play with the ciphers, up to 90-120Mbps with AES-GCM-128 for phase 1.
Forcing AES-CGM seems to be accepted by the main office virtual appliance.
However, it's not possible to use AES-GCM for phase 2, the tunnel won't go up if selected on branch office side.


And then, the final test.

We replaced the main office SFOS appliance (re-read it's specs above) by an APU3D2.
That's an AMD Embedded G series GX-412TC (1 GHz quad Jaguar cores) with 2 GB of RAM.

Then we set up another opnsense on this (less than 100 euros) appliance.

The main (and most interesting difference) with SFOS is that we can force the use of AES-GCM ciphers.
We are also sure AES-NI is used (because it's available both on the XG105 and the APU3D2).
We used, for phase 1:
 . "128 bits AES-GCM with 128 bits ICV" as encryption algorithm
 . AES-XCBC as hash algorithm
And for phase 2:
 . AES128GCM16 as encryption algorithm
 . AES-XCBC as hash algorithm

Same speedtest gives 220 Mbps (up and down, when tested with the FTTH link).

SFOS is supposed to guess/decide if it should use CBC or GCM (https://community.sophos.com/products/xg-firewall/f/network-and-routing/110523/ipsec-aes256-cbc-or-gcm).

We found out 17.5 is not able to deal with GCM for phase 2.

We also found out than forcing the use of GCM (thus AES-NI) enables the XG105 to reach 220 Mbps when using an IPsec tunnel.

It doesn't seem to be the hardware at all (considering the hardware), it's all about software.

We're not aware of any fix/addition about this in SFOS18 (anyway the XG105 won't have it).