Sophos XG 18 MR3 DPI slow download

This is just the way it is.  Also you get the bonus of it breaking random sites with no idea why or what to do about it.

are there any new ideas on this topic?

I don't think that this is not happening to lot's of other people if this would be a bug?

But, what does the XG use?

as I sad, also google, nextdns, whatever... I tried different settings.

And your users?

They use and internal dns resolver which uses nextdns as external. I also tried letting the clients use google or others, no difference and it is just for windows. OSX and Linux are running fine.

So, if you tracert to your dns from a w10 and then compare the result with Linux, what are the differences?

ian

I can't see any difference in a trace., and even if this should not make any difference for a slow download. whie downloading the client will not ask the dns again and again...

You are correct if yo u are downloading one file, but a normal website will have many lookups and XG will need to check each URL etc.

ian

websurfen is not an issue for me, it is only the file downloads...

We have similar issues with DPI.

Downloads run with 2.0-2.5mbits (DualWAN = 1Gbit / 50mbit and 100mbit / 40mbit) with large downloads the download breaks off completely (0kbits in the browser) and the loading of some websites is very slow or not possible at all. Our settings are similar to those described here. If the DPI is deactivated and the web proxy is used, everything runs normally. When the problem really occurs, I can't say anymore. I suspected MR3 currently we are using MR4

Hi fols,

my feelings, but I can't prove it, is that the connection times out but nothing is logged making debugging very difficult.

Ian

Hi fols,

my feelings, but I can't prove it, is that the connection times out but nothing is logged making debugging very difficult.

Ian

We also don't get any error messages from Sophos. But it's definitely up to the HTTPS inspection. There are no problems with HTTP pages. Not even with all HTTPS sites.

For me it is always the same story; I usually enable it following an MR release which promises DPI engine bug fixes and tweaks, then I wait for the calls and e-mails telling me the internet is acting strange or websites don't work.  It usually takes about 12 hours before the problems begin to manifest themselves, and its usually one of two things; either a site won't load (or a particular page within a site won't load), or downloads just don't work, it will download a little bit of a file and then just stop.  Of course the wonderful XG logs all report no problems at all with decryption, nothing being blocked, etc., but the moment you switch the firewall rule back to web proxy, all these mysterious problems vanish.  So there is something wrong with the DPI engine at least in my setup (and its fairly vanilla) and the logging is just too poor to capture it or display anything useful to the administrator as to what is happening or why.  

The only indication of something wrong I ever get in the logs is mounting numbers of "FLOW_TIMEOUT[5] errors.  To my knowledge Sophos has never publically disclosed what this error actually means and honestly I see these errors on sites that aren't having problems and don't see them on sites that do, so I am unsure if it is just a red herring.

This is a very unscientific statement but based on my observations of the problems I have it almost seems to me like the connections get hung or stuck and it results in sites never loading, downloads never downloading, etc.  When I first enable DPI it works and then slowly over the next hours it becomes worse and worse to where it becomes noticeable to the average user out there.

Its a real shame because I want to use it, on paper it sounds wonderful.

Did you every test if this also accours if you have Linux are MacOSX Clients?

For me the issues are there just for my windows machines.

BTW. This also happens if I start a virtual Linux machine on my Windows 10 Desktop. But starting Linux native on the same system with DPI enable there are no issues.

I went through the pcaps with Wireshark for hours without finding the issue :-(

We only have Windows clients in operation. I'm going to test Linux and try it.

@Bill Roland  It is exactly the same with us

Hi,

it happens on MAC clients, MBP, iPad, iPhones.

Mail - iMap/s and smtp/s fail with no error messages.

Ian

I have to admit, that I just testet DPI for HTTP(s) and as this was causing issues I did  not extend the coverage, so I can't talk about imap and smtp.

My current solution is just not to use DPI which is a petty because I would like to make use of it...

Just to confirm, if you use DPI, can you extend the speed of your Line with multiple sessions? For example Ookla uses different downloads at the same time. https://www.speedtest.net/ You can run multiple or single tests. 

Whats the performance with DPI on your appliances? 

right, this works. If i start several downloads I can max out my connection

The reason for the speed difference is because the underlying software on Sophos XG  - who does the DPI is heavily single-core, if you start a download with a single connection the DPI will only be able to use a single core to do almost everything, from the decryption/encryption to IPS and so on.

Meanwhile with multiple connections the DPI Engine is able to share the load with all available cores of your appliance.

The day Sophos updates Snort to 3.1, this issue will be (probably) solved.