Thread Info
  • State Verified Answer
  • +1 person also asked this people also asked this
  • Locked Locked
  • Replies 19 replies
  • Subscribers 30 subscribers
  • Views 5717 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

DPI Engine Bypass

MichaelBolton

Hello,

If I have a firewall rule that has a web policy set to none, so why does the DPI engine still scan the traffic? I thought this was fixed. Still seeing the traffic in the SSL inspection logs. I would really like to reduce the CPU load for traffic I don't want scanned. Running 18.5.1 MR-1, but this has been an issue since the new DPI engine was introduced. I haven't noticed it in awhile since traffic has been low, but I'm now moving a lot of data around and the XG is scanning traffic it shouldn't.

Mike



This thread was automatically locked due to age.

Top Replies

  • +3 verified

    The firewall "sees" all traffic flowing through the XG.  DPI (Deep Packet Inspection) is implemented using snort and it "sees" almost all traffic.

    Snort does many things:
    - detect TLS traffic on any port
    - decrypt TLS traffic if configured to do TLS/SSL rules
    - detect HTTP traffic on any port
    - enforce Web policy on any HTTP / HTTPS
    - enforce ATP
    - enforce IPS
    - enforce app control
    - enforce bandwidth limiting


    If you want the XG to do as little as possible for given traffic, each of these things should be turned off for that traffic. snort will still see the traffic, but it will do less processing. I want to focus on the item that is the most common cause of snort doing more than expected.

    ATP - Advanced Threat Protection - is making sure that a virus on an already infected system cannot talk to any command and control systems on any port. Some parts of ATP are enforced by the DNS server, some by snort rules, and some by web destination rules (by snort or web proxy). It is enabled globally because DNS server enforcement cannot be controlled by firewall rules, and because we want to make sure that ATP protections are enforced on every port.

    When there is no web filer policy, no app filter policy, no a/v scanning, and no ATP:
    There is detection for TLS traffic on all ports, and TLS decryption rules are applied
    There is detection of plaintext HTTP on all ports. However there is no categorization or enforcement of the HTTP standard.

    However when any of those things change (for example ATP is enabled):
    There is detection for TLS traffic on all ports, and TLS decryption rules are applied. The destination SNI is compared against ATP urls.
    There is detection of plaintext HTTP on all ports. If there is HTTP then there is categorization. The destination URL is compared against ATP urls. The HTTP specification is enforced.

    Because web filer policy, app filter policy, and a/v scanning are all set per firewall rule, it is easy an obvious to turn them off for given traffic.
    Because ATP is set globally, it is not obvious how to turn it off for given traffic. You must use an ATP exception which is configured on the Device Console.

    support.sophos.com/.../KB-000038900

    If your really trust the traffic and want minimal inspection then you must
    - set the TLS rule that matches to Do not decrypt
    - set the firewall rule to Web Policy None and malware scanning off
    - set the firewall rule to App Control policy None and IPS policy None
    - set the firewall rule to have an exception for ATP in Device Console

    --------------------


    > It’s crazy to think I can’t create a firewall rule that has no options, to bypass the DPI. You would think setting all options to none would be the answer. Not in Sophos world.

    You set all options in the firewall rule, but the global option for ATP is still in effect.
    Try turning off the global ATP to see if that causes a drop in CPU. If it does, then determine whether you want to disable it globally, or enable it and create exceptions for your firewall rules.

    > If I have inter-VLAN traffic that is encrypted, say SMB traffic, I don't want the firewall to look at it at all. It uses resources even if you have a policy to "Do Not Decrypt".

    You cannot ever have the firewall "not look at it", snort will look at almost all traffic. If you want the XG to do as little as possible you should not use the global setting for ATP, or you should use ATP exceptions.

    You may also want to look at fastpath. Please note: I am not an expert in fastpath.
    The XG series hardware has a virtual fastpath and the XGS series hardware has a hardware fastpath.
    If you are having CPU concerns, please make sure that fastpath is enabled.

    My understanding is the term "fastpath" is often confused to mean two different but related things.
    - with "fastpath" traffic being inspected by snort is processed more efficiently, resulting in lower CPU
    - with "fastpath offload" traffic that snort determines needs no more inspection will have remaining traffic on that connection forwarded without going through snort, resulting in lower CPU


    docs.sophos.com/.../Architecture.html

    Jump to answer
Parents
  • 0

    DPI is not a proxy and has actually nothing to do with Web policies at all.

    DPI Engine is a architecture, which can look out for TLS/SSL traffic and decrypt it. This decrypted traffic will be passed to the needed modules (for example for HTTPS to the proxy) or to the IPS etc. 

    If you do not want to scan it, create a Do not Decrypt rule, which will not decrypt it, but it still "sees" the traffic. You cannot bypass this system without completely shutdown the entire engine, because its build to consider all traffic, regardless of its state. 

    __________________________________________________________________________________________________________________

  • 0 in reply to LuCar Toni

    Well, that's a bad design IMO. If I have inter-VLAN traffic that is encrypted, say SMB traffic, I don't want the firewall to look at it at all. It uses resources even if you have a policy to "Do Not Decrypt". I shouldn’t have to disable SSL inspection globally to make exceptions.

  • 0 in reply to MichaelBolton

    It does not look at this traffic. It simply sees it flowing. What is the issue in your setup beside the performance bottleneck? 

    BTW: its not the SSL Inspection, its the architecture, you are disabling. Look at the fastpath vs slow path. https://community.sophos.com/sophos-xg-firewall/f/recommended-reads/115976/sophos-xg-firewall-v18-xstream---the-new-dpi-engine-for-web-proxy-explained

    __________________________________________________________________________________________________________________

  • 0 in reply to LuCar Toni

    If it looks at the traffic and logs it, it's looking at the traffic. Your answer contradicts itself. It may not be doing anything with it, but it is still looking at the traffic, thus using system resources when it shouldn't. THe performance bottleneck is my issue. There is no way XG can give it's rated throughput numbers if the traffic is encrypted and SSL/TLS inspection is enabled. I'm at 60% CPU usage on an XG230 Rev 2 with a couble hundred megs going through it because alot of the traffic is encrypted. If I disable SSL/TLS, it drops to 30%. That's a big difference for traffic I don't need inspected and is set to "Do Not Decrypt" as you say.

    If I disable SSL inspection, I am disabling using the DPI engine for traffic. The DPI engine is better than the proxy since it looks at all ports. So if I disable it, I am not getting as good of protection, since I am relying soley on the proxy. That's not a viable option.

  • 0 in reply to MichaelBolton

    Can you please past the performance difference numbers into the forum? 

    You are talking about the performance difference in terms of CPU load, but what is the actually throughput decrease? 

    PS: This architecture is also built for XGS hardware, as it highly profit by the NPU. 

    __________________________________________________________________________________________________________________

  • 0 in reply to LuCar Toni

    I don't have an Ixia test bed to test full throughput, that what Sophos is supposed to do. I am just doing simple math. If I'm at 60% CPU usage with a couple hundred megs going through it, how in the world can it give the rated througput of 14,800 Mbps IMIX? It can't unless you disable SSL/TLS Inspection and that is my point. Sophos should have a way to bypass it, periiod.

    PS: I don't have an XGS series. Unless Sophos is going to give me an XGS, that doesn't help me, now does it?

  • 0 in reply to MichaelBolton

    So you do not see any downsides beside a CPU increase and are worry, it could cost significant performance decrease? 

    Because i would disagree to change the entire engine. There are reasons to build this architecture like this implementation (especially going forwards in terms of xStream Architecture and NPU). To cut wholes into this architecture just because of performance bottlenecks, which could "eventually" come up, seems not to be the right call. 

    __________________________________________________________________________________________________________________

  • 0 in reply to LuCar Toni

    It's not a worry, it happens. Once I start pushing lots of encrypted SMB traffic over the firewall, It hits over 90% CPU usage and drops packets. I was saying I don't have a test bed to give you exact numbers on what the throuput is and what the differences are with the SSL inspection turned on or off.

    So you don't see an issue with a firewall being sold with X amount of rated througput that it can't deliver without turning features off?

Reply
  • 0 in reply to LuCar Toni

    It's not a worry, it happens. Once I start pushing lots of encrypted SMB traffic over the firewall, It hits over 90% CPU usage and drops packets. I was saying I don't have a test bed to give you exact numbers on what the throuput is and what the differences are with the SSL inspection turned on or off.

    So you don't see an issue with a firewall being sold with X amount of rated througput that it can't deliver without turning features off?

Children
  • 0 in reply to MichaelBolton

    It drops traffic? This should not happen. Even if the hardware is on the hardware limit, it should not drop, it should delay. You should create a Support ID to get this investigated, if you notice dropping packets. 

    If you notice performance decrease by a small margin, this could be expected. 

    __________________________________________________________________________________________________________________