Sophos XG breaks SSL when connecting to Outlook Anywhere

Hello Friends,

I found this post is more interesting and I want to jump into it. I may answer to your problems at some levels. Other than this, it's a development only. Some might get a clue what is about am I talking. Right?

Actually, to improve user experience, there was an additional development taken into SF OS proxy.  Whenever there is an error because of web request blocked or network error, proxy will try to inform the end user about the error. And yes if your request is HTTPS then proxy will handshake with its own CA (SSL CA) to send the error message.  I guess this is gonna wrong here for some user (might for all)  if https request is blocked because of network error.

In outlook case (most probably Office365), autodiscover domain connection (say for example: https://autodiscover.example.com/autodiscover/autodiscover.xml OR https://example.com/autodiscover/autodiscover.xml) gets failed because of server which sends RST signal, I mean server is not accepting the connection request which leads proxy to send the same "connection error" with 502 response code to end user because proxy is also failed to connect to server and which is resulting into certificate error. 

Note: You may have different reason other then this. But this is what I found at some customer place.  

And the good news is, we have an enhancement in which we are going to revert the changes. That means proxy won't send any error message if there is a network issue.  

This is what all about the issue is but what is the solution?  hold on...hold on...

To resolve this issue, we can have two options.

(1) Install the SSL CA certificate: 

I know this is not a good solution, even I don't think it is a solution especially when I didn't configure SSL scanning.  But for a suggestion, I think it is a good solution. just kidding...

(2) Using FQDN base rule:

Create a FQDN host, in that configure your domain of Office365  like, example.com & autodiscovery.example.com etc...

Create network base security policy and in that configure FQDN host as a destination network.

Note: Don't apply any web filter policy in this rule. 

2nd solution is more helpful instead of disabling whole web filter policy feature. It requires only less configuration.

Hope it clears to you and helpful to you.

Regards,

Vishal Patel

I can confirm this works on XG (Second option)

Make sure you put it before you default outbound rule.

I had similar issues, creating a rule never fully removed the certificate warnings.  We are using Outlook with Office 365 email.

the only solution that worked for me was to deploy the certificate from the XG.  I haven't tested this after the V16 release, only prior.

I had this problem here, solved it creating a rule without web filter BEFORE the rule that have web filter, allowing everithing to autodiscover.mydomain.onmicrosoft.com and autodiscover.mydomain.com.

Hi all,

I am having the same issue with Citrix StoreFront and XG.

And adding the rule without Web filter does not solve the problem.

Even turning off web filtering completely on default rule, does not solve the problem. (It's a test environment only, so I can turn everything off)

When I try to https to StoreFront, I am presented with Sophos Appliance certificate and the connection does not work.

Any help would be much appreciated.

Thanks.

Regards.

I also created the rule with the FQDN of autodiscover.mycompany.com and it's still throwing security certificate errors at me. I did put it before the Default inbound/outbound rule. 

Only way I don't get the certificate errors is if I turn off web filtering all together on every rule. 

we deployed the certificate via group policy to all machines to clear up the message.

its kinda lame that we needed to do that.  The Sophos response thus far has been that is necessary and there exists no way to make it go away. 

The issue with that response is that other vendors don't require this to happen in order to inspect the traffic, this is an XG-specific thing that they desperately need to fix. 

The autodiscover certificate or the Sophos XG certificate? 

The Sophos certificate. 

in our experience, we were getting the error on almost every site when turning on web filtering until the certificate was deployed. 

with filtering turned off, we still got an error in Outlook related to the same problem. 

Hence, we just bit the bullet and deployed the certificate, which isn't that difficult via a GPO

Is there an article or something you used to correctly deploy the certificate? 

https://technet.microsoft.com/en-us/library/cc772491(v=ws.11).aspx

If I recall correctly, in a world without XG:

Outlook tries to do an SSL connection to a site that does not exist (or supports SSL?)

Outlook fails on the SSL connect

Outlook tries to do an SSL connection to the next site, and that works

Outlook is happy.

In a world with XG:

Outlook tries to do an SSL connection to a site that does not exist (or supports SSL?)

The XG intercepts this, can see that it will fail.  It does man-in-the-middle so that it can display a friendly error message.

Outlook gets a successful SSL connect, but cannot use it.

Outlook is not happy.

Try the following:

Go to admin advanced console and type the following command.  This turns off all microapp detection but also affects some HTTPS interception.

system application_classification microapp-discovery off

Do a packet-capture (wireshark) on the client computer

Capture the failure.  Look at exactly what it is trying to connect to.

Create a higher level firewall rule with the destination IP of what you found in the packet capture.  No HTTPS decryption, etc.

Retest.  If it continues to fail, take another look at the capture to find next thing it tries to connect to.

Remember - When the application fails it is probably when the packet capture says there is a successful connection (unfortunately it is to the XG not to whatever it thinks it is going to).  You need to actually let that connection through so that the connection fails, causing the application to go to the next one and succeed.

If I recall correctly, in a world without XG:

Outlook tries to do an SSL connection to a site that does not exist (or supports SSL?)

Outlook fails on the SSL connect

Outlook tries to do an SSL connection to the next site, and that works

Outlook is happy.

In a world with XG:

Outlook tries to do an SSL connection to a site that does not exist (or supports SSL?)

The XG intercepts this, can see that it will fail.  It does man-in-the-middle so that it can display a friendly error message.

Outlook gets a successful SSL connect, but cannot use it.

Outlook is not happy.

Try the following:

Go to admin advanced console and type the following command.  This turns off all microapp detection but also affects some HTTPS interception.

system application_classification microapp-discovery off

Do a packet-capture (wireshark) on the client computer

Capture the failure.  Look at exactly what it is trying to connect to.

Create a higher level firewall rule with the destination IP of what you found in the packet capture.  No HTTPS decryption, etc.

Retest.  If it continues to fail, take another look at the capture to find next thing it tries to connect to.

Remember - When the application fails it is probably when the packet capture says there is a successful connection (unfortunately it is to the XG not to whatever it thinks it is going to).  You need to actually let that connection through so that the connection fails, causing the application to go to the next one and succeed.

that seems like a lot of work to fix something that Sophos should be fixing.  I agree with your breakdown of what is happening.

but seriously, deploying the certificate takes much less time and effort than the troubleshooting you are describing here.

Deploying the Sophos Cert via Group Policy seems to be working for me. 

Finally!!!

Turned off HTTPS decryption on everything. No luck!

Turning off all microapp detection did the trick for me:

system application_classification microapp-discovery off

Thanks Michael Dunn.

The only thing I don't know is how is that affecting other functionality of the FW?

Regards.

If you are not using Application Control, this has no effect.

If you are using Application Control there is an effect.  Application Control uses IPS to watch all tcp/udp stream for certain data.  However IPS cannot watch encrypted HTTPS streams.  "micro-apps" refers to applications that use HTTPS and the web proxy does the application detection instead of IPS.  In v15 - v16.05 turning this flag off has an effect that no applications using HTTPS will be identified and can be blocked.  An example of this would be facebook chat, which uses specific URL paths in HTTPS.

In v17 the current plan is to change the functionality and meaning of this flag, so that in a default configuration most people will not have this problem.

So are you saying if we execute the command  "system application_classification microapp-discovery off" to fix this problem it will break application control BUT it will be fixed in v17?  I.e. we don't have to go back and do anything after the fact?


-Allan

I can confirm this will be fixed in a v16.5 MR, but I don't have a timeline.  In other words before v17.

Application control works by watching streams for traffic.  "microapps" (terrible name) are when the application is using HTTPS, because it is an encrypted SSL tunnel the application detection does not work without the help of the http proxy.

So if you run that command most of application control will work, but application control over HTTPS will not.

If you ran:
system application_classification microapp-discovery off

Then after the fix is in place you do not need to do anything.  The normal behavior in the future will be with it off and all systems that upgrade will turn it off.

Some of the early information on this workaround also recommended doing this (which turns out to be unnecessary):
system application_classification off

If you did this you must manually set it back on.

Thank-you for the reply, I will turn it off and not worry about it since it would be off with the update anyway and should fix the issue in the meantime.

This has not worked for me.  I have turned off the microapp-discovery yet I am still getting the security alert for autodiscover.  Since the "issued by" says Sophos SSL i think the problem is with the router.

Any other suggestions?

The issue is caused by the XG sending the traffic to the Web Proxy, which then does SSL man-in-the-middle in order to show a friendly error message.

One reason for the traffic being sent to Web Proxy is that flag, but there are other reasons.  Most of them have to do with your firewall rules.

Make sure in your firewall rules there is a high-level rule that would apply to this traffic, but that has HTTPS Decrypt and Scan off and Web Policy None.

Thank You Michael!

After spending an extremely long time on this going through Certficate and VPN settings I thoguht I would bite the bullet and try your fix.

Ran the command

system application_classification microapp-discovery off

as suggested and boom, outlook issues stopped instantly as well as an issue I had with HTTPS traffic going across our site to site VPN.