Sophos XG HTTPS Hostname Blocking

Strange behaviour, Oliver.

Are you upgrading from Cyberoam or another XG version?

Also did you try to reboot the XG?

I would recommend you to also regenerate the Certificate and the CA (both built-in) from Certificate menu.

Let us know.

Thanks

Hello,

It's a brand new install of v16. I have tried rebooting by its made no difference. I have also tried regenerating the appliance certificate but I'm still getting the issue.

Regards

Oliver Knights

Oliver,

try to disable the pharming protection under Web Protection > Protection > Malware Scanning > Advanced settings.

I am not sure if there is a way to bypass pharming protection for single ip/url/subnet.

Thanks

Hello 

Disabling pharming protection seems to make it work.

Thanks

Oliver

Thanks Oliver.

sachingurung, Aditya Patel, we need an option inside the XG where the pharming protection can be disabled under Web > Exceptions. Disabling pharming protection at all is not a safe way.

Thanks

Thanks Oliver.

sachingurung, Aditya Patel, we need an option inside the XG where the pharming protection can be disabled under Web > Exceptions. Disabling pharming protection at all is not a safe way.

Thanks

I know Michael Dunn explained some of this in another thread https://community.sophos.com/products/xg-firewall/f/sophos-xg-firewall-general-discussion/84901/web-proxy-always-active-and-interferes-with-dns without going into details as to why. I still don't understand why ALL DNS traffic is being hijacked with pharming protection enabled specially when there are no web protection rules in place.

1)

It is not "all DNS" but "double-check the DNS for all web traffic".  And yes, it occurs even when there is no web protection rules in place.

2)

The fact that transparent mode web traffic goes through the proxy is a design issue that we are aware of and are working on a resolution for, with no ETA.

3)

If you are using web and you do need more granular control of Pharming Protection, please raise it as a feature request.  As with any feature request the more voices, the higher the priority.

4)

If turning off Pharming Protection still does not resolve the problem, and the problem is specifically with HTTPS, and you are not doing anything with Application Control, then there is one additional thing you can do.

ssh into the box as admin (or use Console in the admin menu).

Choose option 4 (Device Console)

system application_classification microapp-discovery off
system application_classification off

This will disable a few things around microapp discovery in HTTPS traffic, affecting both Application detection and the web proxy.

Thanks Michael for your deep info.

For security reason pharming protection should be always ON unless there is a false positive like this case. We need an option on the XG where we can disable pharming proteciton for some URL/IP.

Do we need to open a feature request for it, or there is already something planned for it?

Thanks

There may or may not be a feature request, I don't know.

But I do know that there is no current plans to put in additional configuration for this.

Feature requests that come in via any avenue (Partner feedback, beta, feature request site, etc) are weighted by how important they are - how many (and what size) of customers requesting, workarounds, etc.  The more data that you can add, the higher the priority.  In other words, if this is important to you (over other features) then don't go "oh, they already have a feature request so I don't need to do anything".

Hi Michael and as always thanks for taking the time to explain the thought process. I am not going to argue the pros and cons of FORCED pharming protection but here is my problem with the whole thing.

1. It always ends up with me complaining about logging. Why is the intercepted traffic not logged? Maybe it is being logged and I have never seen it... if that is the case where should I be looking for such logs? Webfiltering? Firewall? System? Moreover, when/if they improve the logging in v17 this traffic will definitely be logged somewhere?

2. Unless you had taken the time to explain this, there is probably nobody else on these boards that knew about this "hidden feature" nor is there any documentation on the actual behavior of pharming protection. From Sophos XG web interface reference guide v16 "Protect users against pharming and other domain name poisoning attacks by repeating DNS lookups before connecting". What does that mean? Repeat a query to a forwarder even if TTL has not expired? Query the root servers? Query a forwarder that is DNSSEC aware? XG/UTM will make a DNS request for any traffic through httpproxy so what is it that the pharming protection does in addition to initial DNS queries if you don't count the unintended DNS hijacking behavior[:D]

3. I realize that I can turn of pharming if it is an issue within my network but that is not what I am arguing. When pharming protection was developed in XG, someone decided to add a DNAT rule for port 53 for all http traffic even when the traffic is not passing the web filtering mechanism. This is clearly a bug and shouldn't need a feature request to fix.

Thanks again for the time that you spend on these boards. I for one really appreciate ALL your feedback.

Regards

Bill 

It is not forced pharming protection.  Its just on by default.
It is applied even if you don't have any web proxying configured.  We're looking into changing that in the future.


2.
So my understanding of the feature is limited, but I think roughly this is what is actually does.

With no pharming protection:
Laptop gets infected with malware.  The malware creates an entry in the hosts files that mybank.com is 1.2.3.4
Laptop browser goes to mybank.com
Laptop makes a TCP connection to 1.2.3.4
XG intercepts the connection and answers it.
On the connection is made the laptop sends an HTTP request like GET /   Host:mybank.com
XG makes a TCP connection to 1.2.3.4 (since that is where the original one was meant to go)
XG sends the HTTP request on that connection
Laptop browser shows that it went to mybank.com but in reality it is going to a pharming site that looks the same and steal the login credentials.


With pharming protection:
Laptop gets infected with malware.  The malware creates an entry in the hosts files that mybank.com is 1.2.3.4
Laptop browser goes to mybank.com
Laptop makes a TCP connection to 1.2.3.4
XG intercepts the connection and answers it.
On the connection the laptop sends an HTTP request like GET /   Host:mybank.com
-->>  XG makes a DNS lookup for mybank.com
XG makes a TCP connection to 5.6.7.8 (which is where mybank.com really lives)
XG sends the HTTP request on that connection
Laptop browser shows the correct website


1.
As far as I know, pharming protection is not logged anywhere.  My limited understanding is that there is no check to see if the original IP the connection was made to was owned by that domain.  Instead it just throws away the original IP and just uses the one it resolves itself.  Trying to do a compare is more costly and may result in false positives for domains with multiple IPs or different IPs based on geolocation.

Therefore it is not a "detected threat" that can be logged.  AFAIK.


3.
I know nothing about port 53 stuff.
To my limited knowledge, turning on/off pharming protection does not affect DNS done on port 53.

Thanks again Michael, your description pretty much clarifies the limited knowledge of pharming protection that I have. ie. A compromised client with hijacked DNS or fake host entries can make connections to fake sites look like a legitimate connection.

However, from what I understand about UTM maybe XG also, when you go via http proxy, this is what roughly happens.

• A client wants to go to cnn.com, the httpproxy will make a Query to the UTM/XG DNS server for cnn.com, fetch the page on behalf of the client. That is why we need certificates for https traffic.
• The DNS server defined in the client's machine has no effect and the hosts file has no effect on http proxy since it is using its own DNS server.

Where I get confused is what extra steps are taken for pharming protection because the basic proxy behavior should mitigate any host files entries etc in the client machine. Obviously, XG is also intercepting DNS requests directly which generally would be a good thing if it was documented. Maybe if the web exceptions section would bypass the DNS interception like  has suggested, that would be a good compromise but that will require another feature request[:D]

AFAIK what you describe is with pharming protection on.

In a normal proxy situation, the UTM/XG does make a dns request to the local server.  It needs this for various protection enforcement.  It also always needs to do this for explicit mode since in that case the browser makes a tcp connection directly to the proxy, not to the internet server.

However AFAIK with the pharming protection off and transparent mode, the IP address the that UTM/XG got from DNS is not used in the actual connection.  Instead the original IP that was used in the TCP connection is used.

HTTPS is slightly more complex and changes whether you have decryption on or not.  But in this regard I think it behaves the same.


Here would be an interesting experiment:
Transparent mode with pharming protection off.
Create a client host entry for doesnotexistindns.com to a valid IP
Now browse to there.
My guess is it will work.  Even though the XG cannot resolve the domain name, it uses the IP from the TCP request to make the connection.
Furthermore, I think it would work whether you have the proxy not explicitly configured, or you are using configured policies.